acceptodds
Under review as a conference paper at ICLR 2027

Better Fixed-State Fidelity Need Not Improve Closed-Loop Performance in Quantized Neural Policies

Abstract

Does closer action agreement with a full-precision policy imply better closed-loop performance after quantization? We hold trained source policies fixed, vary calibration data or compiler settings, and execute compiled policies on Edge TPU and Hailo accelerators in simulated control tasks with paired initial conditions. In a complete, prespecified eight-rule Hailo FetchReach family, one artifact has a 16% smaller fixed-state mean action-error norm than another yet incurs a 34- fold larger estimated mean return loss relative to FP32. An early–middle contrast selected from that family and locked before replication favors early-state calibration in seven of eight independently trained FetchReach policies, and outcomeinformed audits retain large calibration effects under alternative compiler settings. Within the original family, a full-budget, AdaRound-enabled compiler intervention reduces action root-mean-square error (RMSE) on calibration-disjoint states for all eight rules, but mean return decreases for five; retrospective family-adjusted intervals place all five changes below zero but within the prespecified 0.5-return practical band. In a separate prospective benchmark of eight fresh FetchReach policies and 128 Hailo artifacts, a secondary analysis finds lower RMSE on a calibration-disjoint panel in 53 of 64 policy–rule comparisons, 22 of them with lower observed mean return; these are descriptive contrasts clustered within policies. Post-outcome episode resampling retains 16 of the 53 RMSE improvements with adjusted RMSE-change intervals below zero; the count varies across bootstrap seeds. Changing action coordinates or evaluation states changes which comparisons reverse, and reversals also occur when both executables are scored on the same visited states. We provide an execution-bound qualification protocol and provenance-recording harness that compare paired task outcomes against task-specific tolerances. The results distinguish fidelity-based candidate selection from task-tolerance approval and support evaluating the measured executable on its intended execution path rather than fixed-state fidelity alone.

Primary area:
reinforcement learning
Keywords:
post-training quantization, reinforcement learning, closed-loop evaluation, neural policy deployment, execution-bound qualification, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qK53OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper evaluates whether the rank order between quantization (or other approximation) error in a given step of a policy translates to rank order between downstream task performance, finding that this does not always hold. The paper also proposes a test for whether a given policy quantization is within a tolerance by testing numerical error of a igven policy against FP32.

Critical strengths

None

Critical weaknesses
  • The manuscript is heavily (possibly entirely) AI-written and is not very legible.
    • Many terms in the paper are undefined jargon
    • Some plots (e.g., Figure 1) are missing legends.
  • Both contributions reflect existing understanding and practice:
Questions for authors

I would be willing to update my review with substantial clarification of the novelty of the paper

Reviewer_rdEfOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies whether better fixed-state action agreement with an FP32 policy translates into better closed-loop performance after quantization. By varying calibration data and compiler settings while holding source policies fixed, it demonstrates fidelity-return ordering reversals on hardware-executed policies in simulated control tasks. It also presents an evaluation protocol that compares paired task outcomes against predefined tolerances and records the exact executable and execution path.

Critical strengths
  1. The controlled comparisons provide useful evidence about deployment sensitivity. Evaluating actual compiled policies on accelerators, holding source policies fixed, and pairing initial conditions help distinguish construction-dependent behavior from differences in training or evaluation starts.

  2. The paper is careful about the scope of its evidence. It distinguishes statistical significance from practical degradation, candidate selection from deployment approval, and prospective experiments from retrospective analyses. Reporting complete artifact families, negative results, and execution provenance improves the transparency of the empirical study.

Critical weaknesses
  1. I think the core observation/conclusion of this work, which claims that better fidelity need not improve close loop performance, is largely expected and lacks sufficient novelty. Fixed-state fidelity measures local agreement with a reference policy, whereas closed-loop performance depends on how action errors interact with system dynamics and affect subsequent trajectories. A perfect correspondence between the two is therefore not expected. Indeed, Section 4.2 reports a Spearman correlation of 0.833 between error magnitude and performance degradation across the eight Hailo artifacts, showing a strong overall association despite imperfect ordering. The reported reversals illustrate this expected limitation, but the paper offers limited additional insight beyond demonstrating that better fidelity does not always yield better closed-loop performance. I do not find this observation alone sufficiently novel to constitute a compelling core contribution.

  2. The experiments leave the generality and underlying explanation of the fidelity-performance mismatch insufficiently established. The main comparisons concentrate on Hailo and FetchReach, with fidelity varied primarily through calibration and compiler settings and summarized by mean action-error norms or RMSE. Outliers can substantially affect quantization quality, as highlighted by SmoothQuant[1]. yet the main text does not adequately examine their potential role in the observed fidelty mismatch. Presentation also needs improvement: results are scattered across numerous appendix tables, and the main text lacks a concise quantitative summary linking fidelity, task performance, and uncertainty.

[1]: Xiao, Guangxuan, et al. "Smoothquant: Accurate and efficient post-training quantization for large language models." International conference on machine learning. PMLR, 2023.

Questions for authors
  1. What generalizable insight explains or predicts the fidelity-return reversals beyond the observation that fixed-state error is an imperfect surrogate? Given the unresolved mechanism tests and decision-baseline comparisons, what is the principal advance over directly evaluating quantized policies in closed-loop tasks?

  2. Can you demonstrate the same fidelity-return mismatch across additional task and architecture families, including cases where improved fidelity accompanies degradation beyond the predefined practical tolerance? A consolidated table reporting paired fidelity and return changes, uncertainty in both quantities, and tolerance comparisons would help assess the strength and scope of this evidence.

Reviewer_yobZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper evaluates whether lower action error after quantization leads to better closed-loop performance. The author keep the trained policies fixed, vary the calibration data and compiler settings, and test the resulting executables on Edge TPU and Hailo hardware in simulated control tasks. The results show that smaller action errors on the fixed states do not always lead to higher task returns. The Hailo FetchReach calibration experiments show large mismatches, although the return drops in the original compiler comparison remain within the predefined tolerance. The paper also propose an evaluation procedure to see if the differences fall within an acceptable range for each tasks. In this protocol, action fidelity helps screen candidates, while task performance determines whether a deployed policy meets the acceptance criteria.

Critical strengths
  1. The paper covers an extensive experiment sets across multiple tasks, hardware backends, and compiler settings. Replication with multiple experimental details are well provided in the supplementary material and appendices, providing substantial empirical analysis.

  2. The paper addresses a well-motivated and practically relevant question, whether a quantized policy performs better in closed-loop tasks when its action are closer to those of the full-precision policy. The concern is intuitive, and examining it on real deployment hardware is worthwhile.

Critical weaknesses
  1. The manuscript is lack of the overall workflow which could help reader link every component together easily. For example, Section 3 introduces rebuild results, task-specific setting and other definitions; however, Section 4 and appendices later introduce the experimental details which is slightly mentioned in the Section 3. Therefore, if reader wants to understand the protocol, they need to go through other sections and read back Section 3.

  2. Although the paper provide credible counterexamples, the most obvious evidence of practically significant mismatches comes from Hailo FetchRench. The first eight-policy experiment mainly shows that changing calibration data affects task performance, and the separate experiment with new policies continue to test whether lower action error can come with worse task performance, but it still remain on the same tasks. Also, all five drops in return in the original compiler comparison stayed with in the paper's tolerance, which shows that lower action error does not always mean better task performance. Therefore, more direct comparison across tasks and policy architecture would help here to show whether large mismatches is a general observation.

Questions for authors
  1. In the introduction section, there are several constraints introduced before the main contribution, which might reduce your original contribution/s strengthen. Also, the conclusion could provide a more balanced discussion of the findings and their potential limitations.

  2. It would better to reorganize the related work sections to provide the reason why we need this proposed evaluation study. Keep comparisons focused on the specific difference from other existing work and moving detailed experimental settings and results would improve the readability.

  3. The Hailo compiler mentioned L0 - L3 could be slightly mentioned in the beginning for reader to easily follow.

  4. Both the CI rule and the simpler fixed-margin baseline show no-observed conflicts with resolved holdout outcome. It would be better to clarify additional practical benefit the CI rule offer?

  5. Based on the 22 reported cases, it could be better to show statistically significant decreases in both action RMSE and task return by assessing two changes jointly and account for multiple comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.