acceptodds
Desk rejected at ICLR 2027

GLaCS: Grounded Constraint-State Projection for Global-Consistency Reasoning

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Global-consistency reasoning requires an answer supported by a joint assignment that satisfies interdependent constraints. We study how to select such an answer from a fixed candidate set. We propose GLaCS, a grounded constraint-state projec- tion framework that separates candidate generation from structured authorization. GLaCS extracts variables and relations from the input and compiles them into executable state computations and factorized consistency scores. Task adapters use constrained assignment search, shared numerical repair, or deterministic execution to score the fixed candidates. The lowest-energy candidate is selected through a shared scoring and intervention interface. On identical reflection self-consistency pools, GLaCS improves accuracy from 0.965 to 0.995 on arithmetic chains and from 0.658 to 0.775 on logic-grid transfer. On four extended BIG-Bench Hard tasks, LLM-extracted graphs yield 0.941 accuracy, compared with 0.711 for an LLM selector given the same graphs. Component interventions, graph perturbations, and stem-only extraction audits characterize the roles of state repair, constraint fidelity, and candidate coverage in these gains.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Constraint reasoning, Neuro-symbolic reasoning, Energy-based inference, Structured prediction, Large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LaS4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how to select an answer from a fixed candidate pool using constraints extracted from the input. GLaCS converts these constraints into task specific states and factors, computes candidate scores through numerical repair, assignment search, or deterministic execution, and selects the candidate with the lowest energy. Experiments on arithmetic chains, logic grids, and BIG-Bench Hard report improvements over prompting and LLM selection baselines. Additional analyses examine candidate coverage, extraction quality, and the effects of constraint interventions.

Critical strengths
  • The paper addresses the important challenge of improving the reliability of language model reasoning.

  • The experiments evaluate the method across several reasoning tasks and report improvements over the baselines, demonstrating its effectiveness on the evaluated tasks.

Critical weaknesses
  • GLaCS relies on predefined adapters for the evaluated task families, including their representations, relations, and executable operations. However, the prompting and LLM selection baselines generate or select answers through model inference, without the predefined executable procedures used by GLaCS (Appendix A, “Prompting and decoding”; Appendix B.3, Table 22; Appendix B.5, Table 28). Therefore, GLaCS makes a stronger assumption about the availability of solution structures and operations. This limits its generality, as extending it to new task families requires designing adapters that define both problem representations and solving procedures, which can be difficult.

  • In Section “Grounded Constraint Construction”, the authors state that Normalize resolves entity references and task specific relations into a shared outer interface. However, the normalization rules and implementation are insufficiently specified, including how references are matched, relations are converted, and records are rejected. This makes the conversion from extracted records to executable constraints difficult to assess.

Questions for authors
  • How is Normalize implemented to resolve entity references and convert extracted relations into the shared interface?

  • How can GLaCS be extended to new task families, and what adapter design is required?

2 comments
AuthorsOpenReview ↗

Normalize implementation and extending GLaCS to new task families

Thank you for the review. We answer both questions below.

Q1. How Normalize is implemented. Normalize is deterministic. It maps each family's extracted records onto one shared interface of entities, typed relations and a query.

Entity references are matched in three stages. We first strip articles, lowercase the text and normalize punctuation and whitespace. A reference then resolves by exact match on the normalized string, by substring match, or by the highest string similarity above a threshold of 0.78, tried in that order. If two entities still tie, declaration order decides.

Relations are converted per family. Logical ordering compiles to fixed-position or relative-order predicates, and tracking to initial assignments followed by an ordered list of swaps. Temporal turns times into minutes and busy intervals, navigation becomes ordered moves and turns, Web of Lies becomes ordered truth links, and Boolean inputs become expression trees.

Records are rejected at two levels. A record that cannot be coerced is skipped on its own. A graph that lacks a required field or a valid query is a parser failure and goes to the same fixed fallback rule that the matched solver and execution controls use.

We logged every raw record, its normalized form, the matching path and any rejection reason for all 640 headline BBH graphs. Of 3,715 records, 3,630 are accepted and 85, or 2.3%, are rejected: 82 in logical ordering and 3 in navigation. No graph is lost.

The trace also shows that the headline numbers are conservative. The position parser rejects bare ordinals such as "third" or "last". We fixed a compatibility rule in advance that maps them to positions; it changes 44 of the 192 logical graphs. Accuracy rises from 149/192 to 163/192, a gain of 7.29 points with a paired 95% CI of [3.13, 11.98] and p = .00258. Z3 and enumeration still agree on every candidate energy. The headline tables keep the original protocol, so any gain from better normalization comes on top of the reported results.

Q2 and W1. Adapter design and new task families. Making the task semantics explicit is the design choice the paper evaluates. GLaCS authorizes a candidate by executing the declared relations, and the other selection methods instead generate an answer. Two controls make this a fair test. The graph-listwise selector receives exactly the same extracted graph and candidate pool as GLaCS, so it has the same access to structure. The shuffled-graph controls keep the executor fixed and break the alignment between graph and question, which tests whether the gain comes from the correct grounded state.

The language-to-solver work the paper builds on makes the same assumption: SatLM and Logic-LM also rely on a predefined solver language with fixed semantics.

A new family needs a state domain with its extraction schema, relation normalization, executable factors, a map from query to answer, a candidate compatibility check and a failure and tie policy, plus tests for valid, invalid and contradictory inputs. Candidate scoring, selection, the interventions and the coverage and extraction audits are shared and reused without changes. For relations we already support, the generic constraint DSL reproduces the task-specific translators' decisions on 448 examples across Temporal, Boolean, Tracking and logic grids, as reported in Table 31.

An adapter does not have to implement its own search either. The GLaCS energy is a standard minimum-violation objective, so an off-the-shelf solver can compute it. Independent Z3 encodings on identical graphs, candidates, weights and tie rules reproduce every GLaCS candidate energy and decision on 192 of 192 BBH logical examples and on 512 of 512 generated-candidate logic grids. On structured logic grids, Z3 solves all 96 instances at 4, 5, 6 and 8 entities. The 8-entity grids have 65,548,320,768,000 assignments and take a median of 0.3267 s per example. Most of the per-family effort therefore goes into declaring semantics rather than writing search code.

We will release the code together with the adapters, the extraction and selection prompts, raw predictions, normalization traces and frozen configurations, with source snapshots and input hashes.

Thanks to the authors for their response. However, supporting a new relation vocabulary requires an adapter that defines its executable semantics. The need to design such adapters for new task families limits the method’s generality. For this reason, I will keep my score.

Reviewer_uPYmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GLaCS, a framework for selecting answers from a fixed candidate set using grounded constraint representations. The framework extracts entities and relations, compiles them into task-specific states and factors, and selects the candidate with the lowest consistency energy. Its implementations include numerical repair, exhaustive assignment search, and deterministic execution followed by candidate checks. Experiments on arithmetic chains, logic grids, and selected BIG-Bench Hard tasks show substantial improvements over prompting and language-model selectors. The paper also provides extensive audits of candidate coverage, extraction fidelity, option visibility, graph perturbations, and computational cost. However, the experiments do not sufficiently distinguish the proposed framework from conventional constraint solving or direct execution followed by candidate matching.

Critical strengths
  1. Clear experimental separation of generation and selection. Fixing candidate pools helps distinguish improvements in selection from changes in candidate coverage. Reporting coverage ceilings and conditional selection accuracy makes the results easier to interpret.
  2. Evaluation with learned constraint extraction. The paper goes beyond idealized structured inputs and measures performance using LLM-extracted graphs. The comparison against graph-listwise selectors supports the usefulness of executable constraint evaluation rather than merely providing structured information to an LLM.
  3. Substantial diagnostic evidence. Stem-only extraction, graph-induced candidates, semantic fidelity measurements, shuffled-graph controls, multi-seed evaluations, paired tests, and timing measurements address several plausible alternative explanations.
  4. Transparent discussion of limitations. The paper acknowledges dependence on extraction quality, candidate coverage, and task-specific adapters. The broader benchmark results also help identify where the approach currently provides limited benefit.
Critical weaknesses
  1. The most relevant solver-based comparisons are missing. The paper builds on established language-to-symbolic-solving approaches, including SatLM and Logic-LM, but its main comparisons emphasize prompting and generative selectors. Providing the same graph to an LLM selector is a useful control, yet outperforming that selector does not establish an advantage over conventional executable constraint solving.identify the methodological contribution beyond a unified interface for task-specific symbolic execution.

  2. The independent contribution of candidate-preserving projection is unclear in several task implementations. Tracking, navigation, truth propagation, and Boolean evaluation primarily execute the extracted graph and check whether each candidate agrees with the result. Logic-grid evaluation enumerates assignments and groups their minimum violation scores by answer.

  3. The direct-execution comparison does not use equivalent answer requirements. The Extended aggregate reports 0.691 for direct graph execution versus 0.941 for GLaCS. Appendix Table 48 shows that the direct executor abstains on every Temporal example because the graph does not identify a unique answer interval. GLaCS instead evaluates the supplied candidate intervals for feasibility.

  4. Evidence for scalability and generalization beyond the supported adapters remains limited. The reported logic grids use four entities and three attribute categories, with exhaustive enumeration of 13,824 states. The search space grows factorially, so the measured runtime does not establish scalability to substantially larger instances. Supporting new relation vocabularies also requires adapters defining their executable semantics; the generic DSL audit demonstrates implementation agreement rather than transfer to unseen semantics.

Questions for authors
  1. Can you compare GLaCS with conventional constraint or minimum-violation solvers using identical extracted graphs, candidate pools, and tie-breaking rules? What capability does GLaCS add beyond these baselines?
  2. In which evaluated tasks does candidate-conditioned projection outperform direct execution followed by candidate matching? Can this distinction be isolated experimentally?
  3. Can the Temporal direct-execution baseline evaluate the same supplied options against its computed feasible intervals, rather than being required to generate a unique interval?
  4. How do accuracy and runtime scale with larger assignment spaces? What manual implementation is required to support a previously unseen relation family?
  5. What experimental differences explain the main-text/appendix discrepancies identified above? Can the results be mapped to explicit configurations and evaluation subsets?
2 comments
AuthorsOpenReview ↗

Solver controls, candidate conditioning, scaling and table configurations

Thank you for the careful review. We answer each question below.

Q1. Conventional minimum-violation solvers on identical inputs. Our independent Z3 encodings use the same normalized graphs, candidates, weights, tie rules and fallback as GLaCS. For BBH logical the objective is min[10 × graph violations + candidate mismatch]. For logic grids, Z3 minimizes graph violations with the candidate owner fixed.

ControlnAccuracy, Wilson 95% CIEnergy and decision agreement
Z3, headline BBH logical192149/192 = 77.60% [71.20, 82.93]192/192
Z3, generated-candidate logic grids512397/512 = 77.54% [73.73, 80.94]512/512

Every candidate energy agrees across the two backends, so GLaCS computes the exact optimum of its objective. On the logic grids, GLaCS selects the correct answer in all 397 pools that contain it. All other errors come from pools without a correct candidate, 48 of which are empty. Selection is therefore exact, and the remaining errors are coverage.

What GLaCS adds is the fixed-candidate decision problem and the analysis built on it. Every candidate gets a graded energy, and selection, abstention and grounded interventions are defined on those energies. This is what lets the paper separate coverage from selection, perturb and shuffle graphs, and compare against LLM selectors given the same information. SatLM and Logic-LM solve for one answer and do not score candidates. Our Z3 runs are solver controls on our frozen graphs, not end-to-end reproductions of those systems.

Q2 and Q3. Direct execution, candidate conditioning and Temporal. We gave the Temporal executor the same supplied intervals and checked each one against the same day bounds and busy intervals. It then matches GLaCS's candidate energies and decisions on all 64 Temporal examples, with 63/64 correct and a Wilson 95% CI of [91.67, 99.72]. Same-option execution also matches GLaCS on all 256 Extended examples, 241/256 or 94.14% [90.56, 96.42], and on all 192 Tracking examples, 180/192 or 93.75% [89.39, 96.39].

This control shows that candidate conditioning is what makes Temporal solvable. The constraints do not determine a unique interval, so an executor that must produce one abstains on all 64 examples. Testing the supplied intervals solves 63 of them. The .691 vs. .941 comparison therefore measures the value of a candidate-conditioned answer domain. With the same option requirements, execution computes GLaCS's candidate energy and gives the same result.

Tracking, navigation, truth propagation and Boolean evaluation execute a unique state and score each candidate's agreement with it. Logic grids minimize violations over candidate-compatible assignments, and arithmetic chains use shared-reference numerical repair. Because all of them share one candidate-energy interface, one set of interventions and audits applies to every family.

Q4. Scaling and new relation families. Any exact solver can compute the GLaCS energy. With the Z3 backend we evaluated grids with three attribute categories, 32 examples for each of seeds 7103, 7203 and 7303 at every size. There were no timeouts.

EntitiesAssignmentsAccuracyMedian / p95 s per example
413,82496/96.0304 / .0339
51,728,00096/96.0636 / .0740
6373,248,00096/96.1092 / .1255
865,548,320,768,00096/96.3267 / .3517

At 4 entities, exhaustive enumeration matches every Z3 candidate energy. Switching backends therefore leaves decisions unchanged while the assignment space grows by nine orders of magnitude. These grids use structured generator constraints with every person as a candidate.

A new relation family declares four things: its state domain, normalization, executable factors, and query and candidate compatibility. It also needs failure and tie tests. A standard solver can then compute the energy, and selection and audits are shared. Table 31 shows that the generic DSL reproduces the adapters' decisions for the relations we support.

Q5. Main-text and appendix differences. Every difference maps to a specific source report, tie rule, candidate pool or evaluation subset. In the paired Core and Extended tie audits, no candidate energy changes.

In Extended, the stable canonical tie rule used in Table 1 changes two tied decisions, Temporal from 64 to 63 and Web from 57 to 58, so the total stays at 241/256. The Extended baselines of 114/256 and 125/256 come from two archived reports that differ only on navigation, 26 against 37 correct, which accounts for the whole gap between these unprojected rows. Grid .621/.650 is the legacy report and .645/.658 the later canonical-tie audit, which is the reference for that protocol. The grid main study uses seed 7103 with 128 examples per transformation, the stability study three seeds with 512 each, and the timing tables come from separate timing runs.

Dear Reviewer,

We sincerely thank you for your thoughtful review and constructive feedback. In our rebuttal, we have made every effort to address your concerns by clarifying the target construction and providing additional experimental evidence on component ordering, computational efficiency, and generation performance.

As the discussion period draws to a close, we would be very grateful if you could kindly review our responses and share your thoughts. Should any concerns remain, we would be more than happy to provide further clarification.

Thank you again for your consideration and support!

Best regards,

Authors

Reviewer_o7GEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • interesting paper on global consistentc. makes a constraint graph and selects the candidate answer most consistent with a feasible state.
  • Uses task-specific execution or enumeration for arithmetic chains, logic grids and BBH tasks.
  • Reports gains on shared candidate pools, including controls with shuffled graphs.
Critical strengths
  • The paper provides useful evidence that executing grounded constraints improves answer selection. Shared candidate pools and shuffled-graph controls strengthen this conclusion.
Critical weaknesses

Main:

  • The contribution beyond parsing and executing a graph is not established. Should compare against other baselines.
  • Reconcile the differing projection, fallback and ablation results across the main tables and appendices.

Secondary:

  • Identify the models used for the older BBH baselines and strengthen the listwise selector's decoding budget.
  • Release the missing BBH adapters, prompts and predictions. Clarify labels that describe prompted extraction as learned.
Questions for authors

See above.

2 comments
AuthorsOpenReview ↗

Solver baselines, table reconciliation and stronger LLM selectors

Thank you for the constructive review. We added the baselines you asked for and reconciled every table.

Contribution beyond parsing and executing a graph. GLaCS formulates fixed-candidate authorization. Each candidate receives a violation energy, and selection, abstention and factor-level interventions are all defined on that energy. This lets the coverage and extraction audits separate generation errors from selection errors. To test the executor itself, we ran solver and execution baselines with identical graphs, candidates, weights, tie rules and fallback.

Control on the same inputsnControlGLaCSEnergy and decision agreement
Z3 minimum violation, BBH logical192149149192/192
Z3 minimum violation, logic grid pools512397397512/512
Execution and candidate matching, Tracking and Extended448421421448/448

Full agreement means GLaCS computes the exact optimum of its objective regardless of backend. On the logic grids it picks the right answer in all 397 pools that contain one, so every remaining error is a coverage error. A solver backend also makes the formulation scale: Z3 solves all 96 structured grids at 4, 5, 6 and 8 entities, with a median of 0.3267 s per example on 65,548,320,768,000 assignments. These are solver controls rather than end-to-end SatLM or Logic-LM runs. Those pipelines return a single solver answer, whereas GLaCS ranks every candidate when constraints are partly inconsistent and supports factor-level interventions on any backend.

Main-text and appendix values. Each difference traces to a tie rule, prediction snapshot, candidate pool or subset, and no candidate energy changes in the paired tie audits. In Extended, Table 1 applies the stable canonical tie rule that postdates Tables 35, 38 and 40. It flips two tied decisions, Temporal 64 to 63 and Web 57 to 58, leaving 241/256. The .445 prompted baseline and the .488 no-projection row come from two archived unprojected reports, 114/256 and 125/256, which differ only on navigation, 26/64 against 37/64, so that gap carries no information about projection. Grid .621/.650 is the legacy report and .645/.658 the canonical-tie audit, which is the reference. The stable rule reproduces Core exactly at 329/384 with all 384 energies matching the cached run. Grid .775 and .764 are seed 7103 with 128 examples per transformation against three seeds with 512 each. Chain .680 and .808 use the CoT-SC and Refl-SC pools, and .808 is the matched comparison. Timing tables come from separate timing runs.

Models and decoding budget. The earlier BBH candidates come from Qwen2.5-VL-3B-Instruct with text input. Graphs come from prompted text-only Qwen2.5-3B, 7B and 14B-Instruct extractors, which is what "learned extraction" refers to. Both selectors are Qwen2.5-14B-Instruct with greedy decoding and seed 9401. We then strengthened the selectors in two steps on the same frozen inputs, the independently preserved compute graphs, on which GLaCS scores 240/256 in Extended.

Selector conditionSetGLaCSListwiseGraph-listwiseSelector minus GLaCS, paired 95% CI, listwise; graph-listwise
One-line prompt, 256, 512 or 1024 tokensExtended 256240176179[−30.9, −19.1]; [−29.3, −18.0]
Step-by-step reasoning, 1024 tokensExtended 256240246247[−1.17, +5.86]; [−0.01, +5.86]
Step-by-step reasoning, 1024 tokensCore 384329335330[−2.86, +5.99]; [−4.43, +5.21]

Raising the ceiling changes nothing: one-line outputs are identical at all three budgets and never exceed 80 tokens. With reasoning, the selectors produce about 270 tokens per question, 271.95 and 269.14 on average, yet none of the four comparisons is significant, with exact McNemar p of .286, .143, .572 and 1.000. GLaCS is statistically indistinguishable from them without generating anything, and its decisions have two properties the selectors lack. They do not depend on option order. Across three fixed permutations of the 256 Extended questions, GLaCS keeps its answer in all 768 evaluations, while the reasoning selectors change answers in 33 and 24 of 768, 4.30% [2.34, 6.64] and 3.13% [1.43, 5.08], or 23/758 and 19/763 when both outputs parse. Replaying the original prompts reproduces their outputs 286 of 286 times, so presentation order alone causes the changes. They are also cheap: a GLaCS decision takes a median of 56.32 µs on one CPU thread, against 12.44 s per reasoning prompt on the GPU, both measured for selection after graph extraction.

Reproducibility. We will release the code together with the BBH adapters, prompts, frozen inputs, raw predictions, configurations and normalization traces for all 640 headline graphs, with verified source and input hashes.

Dear Reviewer,

We sincerely thank you for your thoughtful review and constructive feedback. In our rebuttal, we have made every effort to address your concerns by clarifying the target construction and providing additional experimental evidence on component ordering, computational efficiency, and generation performance.

As the discussion period draws to a close, we would be very grateful if you could kindly review our responses and share your thoughts. Should any concerns remain, we would be more than happy to provide further clarification.

Thank you again for your consideration and support!

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.