acceptodds
Under review as a conference paper at ICLR 2027

QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization

Abstract

Four-bit post-training quantization can reduce the memory demands of large language models, but preserving accuracy under strict MXFP4 W4A4 requires coordinating several design choices. Coordinate transforms change block-encoding errors, which in turn affect the residuals propagated through the network. The useful algorithmic decomposition is therefore not fully known before search. LLM-driven program evolution offers a way to explore these choices, but performance scores alone do not explain which design should change next. We introduce QuantForge, a PTQ discovery system that records competing explanations, selects controls that distinguish them, and checks that successor code implements the resulting conclusions. This residual compilation guides program revisions while retaining useful programs even when their original explanations are rejected. Remeasuring the revised program reveals the next error to address. This process discovers HiRes, a fixed MXFP4 quantizer that shapes coordinates, refines legal code assignments, and recovers errors along attention and MLP paths. Each stage acts on residuals measured after the preceding stage has executed. Across seven tasks, HiRes achieves the lowest seven-model Robust Fit (0.09300) and the lowest quantized Fit-7 at 32B. In matched-budget comparisons of LLM-driven program evolution, each with 240 evaluator calls, QuantForge reaches a held-out transfer target in six of eight runs, compared with three each for textual memory and reflection memory, and one for score-only evolution, despite evaluating fewer new programs. These results show that QuantForge improves the discovery of transferable PTQ algorithms by turning controlled evidence into subsequent program changes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Machine Learning, Large Language Models, Model Efficiency, Automated Algorithm Design

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gYRPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes QuantForge, an LLM program search framework for training quantization algorithms after discovering MXFP4. This method maintains candidate programs and unresolved design hypotheses, distinguishes competitive interpretations through comparative experiments, and requires subsequent program modifications to implement experimental conclusions.

The author further proposes the HiRes quantization method discovered by QuantForge, which combines geometric transformation, discrete coding correction, and Attention/MLP error recovery.

Critical strengths

The author used paired random seeds and a unified evaluator budget in the search experiment, and analyzed the core components through program updates, replay, and checker ablation. Compared to only reporting the final optimal program, these experiments provide more specific mechanism evidence.

HiRes achieves lower quantization performance loss on multiple models. The paper also reports the results of stage ablation, different calibration seeds, and the computational cost of BF16 backed implementation.

Critical weaknesses

QuantForge combines existing mechanisms for program evolution, reflection, hypothesis testing, and execution checking. The paper refers to this process as Progressive Residual Factorization, but the current formal description mainly involves the management of unresolved hypotheses, discriminative control selection, and code modification.

Equation (2) can distinguish the prediction of registration interpretation, but cannot guarantee the recognition of the true error mechanism. The existing ablation has demonstrated the effectiveness of a complete search system, but the contributions of correct residual interpretation, program quality control, and invalid candidate filtering have not yet been fully separated.

The search experiment mainly compared Score only, TextMem, and ReflectMem, without directly covering more complete program search frameworks such as ReEvo and AlphaEvolve. All experiments used the same GPT-5.6 Sol proposer and were not validated across other algorithm discovery tasks.

In addition, although HiRes has good simulation quantification accuracy, its native MXFP4 deployment benefits have not been directly verified. Therefore, there are still limitations to the scope of application of the two contributions.

Questions for authors

How to determine the true error mechanism corresponding to an algorithmic residue and successfully solve it? Is there an independent mechanism for verifying the correctness of registration, apart from excluding competitive interpretations?

If traditional program search methods use the same execution checks, invalid candidate filtering, and code revision constraints, but do not use competitive interpretation and residual frontiers, does QuantForge still have significant advantages?

Does QuantForge's results rely on the reasoning ability of GPT-5.6 Sol? Will different proposer LLM or other program search tasks yield similar conclusions?

How to distinguish the numerical advantages of HiRes three-stage design itself from the independent contribution of QuantForge's discovery of this stage structure?

Reviewer_CqssOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduced QuantForge to discover PTQ algorithms through progressive residual factorization, which produced HiRes, whose quantization stages operate on successively realized model states. This paper demonstrates the performance and efficiency of QuantForge and HiRes through various experiments.

Critical strengths
  1. Translating experimental conclusions into subsequent code revisions creates a verifiable search loop. QuantForge goes beyond selecting candidates by score: it uses controlled experiments to distinguish competing explanations and checks whether subsequent code actually implements the resulting conclusions. Comparisons under matched evaluation budgets and targeted ablations provide evidence for the value of this mechanism.
  2. The paper carefully examines dependencies between components. Beyond removing individual components, the authors compare fresh versus stale calibration statistics and substitute alternative geometric transforms while keeping downstream refinement and recovery procedures fixed. These experiments help distinguish the sources of improvement and support the design principle that later corrections should target the actual residual errors produced by earlier interventions.
Critical weaknesses
  1. The connection between QuantForge and the final HiRes algorithm is not sufficiently established. I think the QuantForge’s main workflow is broadly applicable to algorithm optimization, whereas HiRes comprises a set of specific quantization designs. Although the independent search experiments and stage-level statistics demonstrate the framework’s potential, the paper does not sufficiently trace how the key designs in the final HiRes emerged from competing explanations, controlled experiments, and mandatory code responses. Consequently, it remains difficult to assess how much of HiRes’s performance is attributable to QuantForge’s distinctive mechanisms, and how much reflects manual design or subsequent composition. Please provide the key discovery trajectories leading to the final HiRes, explicitly linking experimental verdicts to algorithmic changes, and compare the discovery of these mechanisms against the strongest ReflectMem baseline.
  2. The novelty boundary of HiRes and the attribution of AI and human contributions are unclear. Although the paper cites prior techniques such as transform-based PTQ and GPTQ, it does not systematically distinguish which key components of HiRes are inherited, adapted, newly combined, or newly introduced. This limits assessment of both HiRes’s novelty relative to prior work and AI’s contribution to the new designs. Please disclose external knowledge access and human interventions, such as the literature, external code, and initial design guidance available to or actually used by QuantForge.
  3. The execution architecture and the gains over the underlying general-purpose coding agent are insufficiently specified. The paper reports the proposer model and an abstract control flow, but does not clarify whether QuantForge is an independently implemented search harness, a prompt or skill extension to an existing coding agent such as Codex, or a combination of these. It also does not distinguish externally enforced mechanisms from instructions followed by the model. The implementation choice alone does not determine scientific value, but these details are necessary for reproducibility and attribution of the gains. The Score-only, TextMem, and ReflectMem comparisons are useful, yet it is unclear whether they adequately represent the underlying agent’s ability to autonomously perform the same optimization task. Please supply the actual schemas, key prompts and probe construction, and provide a base-agent comparison with matched model, data access, and resource budgets, or justify why the existing baselines adequately cover it.
  4. It remains unclear whether HiRes’s accuracy gains justify its additional execution costs. HiRes combines coordinate transforms, weight reconstruction, dynamic activation-code refinement, and structural recovery, making separate cost comparisons against existing online activation quantization and offline weight quantization methods necessary. Although the appendix provides some timing and storage analyses, a systematic comparison against strong baselines under matched models, quantization formats, calibration data, and implementation backends is still missing. Please report accuracy jointly with these costs and quantify each component’s benefit and overhead.
Questions for authors

Together with discovery records for HiRes’s designs, can this comparison establish which new designs and performance gains are attributable to QuantForge’s distinctive mechanisms?

Reviewer_zoRHOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents:

  1. QuantForge: An LLM-based solution that searches for the best MXFP4 W4A4 (4-bit weights and 4-bit activations) quantization algorithm.
  2. HiRes: a qunatization algorithm proposed by QuantForge.

Authors present results comparing QuantForge with more basic LLM search and show they achieve better losses after a certain number of calls, and also present results for HiRes showing they achieve better losses than a wide range of existing quantization solutions.

Critical strengths
  • Evaluated on a wide range of tasks and models of various families and sizes
  • Compared with a wide range of methods and showed it beats them on most models
  • Ablated different components of the proposed solution
Critical weaknesses
  • It is not clear in the comparisons table if approaches that uses MXP4 were compared against. Some of the approaches listed in Table 1 are known to be INT4.
  • Paper really difficult to read.
    • Many sentences are incoherent. They sound more like bullet points rather than a smooth flow explaining one concept after another.
    • Terms are used without defining or explaining them.
    • It seems that Robust Fit and Fit-7 are metrics introduced by the paper, but instead, their values are reported in the Abstract, and later in the list of contributions in line 114, as if they are well known standard metrics. Authors should have mentioned that these are metrics they are introducing.
    • Figure 1: it is not clear to me if the figure has anything to do with quantization
    • terms like "program", "candidates", "controls", "scores", "contract", etc. are not clear to me in the context of quantization. I think each should be clearly defined.
    • Section 5.1: The writing language describes steps in the present tense. It is not clear if each step is proposed by the author or borrowed from another approach.
    • Algorithm 1 boc: the algorithm should be self-contained as much as possible, and hence variables like T,D should be defined

It cannot be published in a top-tier conference (or any conference) in that state. I encourage the authors to rewrite the paper and resubmit to another conference.

Questions for authors
  • Can you add columns to Table 1 to indicate the numeric type (e.g., INT4, FP4), as well as tensors (e.g., weright-only vs weights and activation), to each quantization method. I am not sure which of the methods being compared against use MXFP4. A fair comparison
  • Line 191: what is e'?
  • P in line 236 seems to denote coordinates while in line 207 it seems to denote programs
  • Line 454: "The PPL ≤ 8.00 target is reached in 6/8, 3/8, 3/8, and 1/8 runs". It is not clear to me what the notation "x/8" refers to.

Comments on the paper

1 comment
Area_Chair_BPhrOpenReview ↗

Action Required: Respond to Author Rebuttals Before the Discussion Period Ends

Dear Reviewers,

The authors have posted their rebuttals. As the author–reviewer discussion period is coming to a close soon, please read their responses and participate in the discussion.

All reviewers are expected to respond, at minimum with a brief acknowledgment confirming that the reviewers have read and considered the authors’ rebuttals. If your concerns have been addressed or remain unresolved, please briefly indicate this in your response.

Thank you for your time and effort.

Best,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.