acceptodds
Under review as a conference paper at ICLR 2027

PCS-AC: Compiling Output-Sensitive Actions to Correct Medical Predictions

Abstract

Correcting a frozen model for medical decision-making requires interventions that fix errors without disrupting predictions that are already correct. A shared representation edit is often insufficient because the same direction may help some inputs but harm others. We introduce Predictable Corrective Subspaces via Action Compilation (), an output-sensitive framework that learns input-dependent corrective interventions in hidden representations. Specifically, a budget-weighted Jacobian identifies an output-sensitive subspace, within which source labels define bounded corrective actions. We compile these actions into an input-dependent low-rank controller via reduced-rank regression and refine it through the frozen backbone to improve predictions while preserving source margins. At inference, the controller uses only the current hidden state, without labels, gradients, or an additional backbone pass. On 2,100 retrospective radiologic questions, reaches 80.95% and 81.62% accuracy with Qwen2.5-VL-7B and InternVL3.5-8B, improving Native by 12.19 and 42.67 percentage points. Under matched source supervision, response subspace, and refinement budget, it outperforms a learned shared direction by 13.31 and 28.45 points. Matched-capacity comparisons further favor output-sensitive construction over random and activation-PCA pipelines. An independently fitted text controller reaches 95.03% on 3,360 controlled synthetic assertions, a 5.98-point improvement over Native. These results show that compiling output-sensitive corrective actions into conditional controllers enables more selective correction of frozen models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Activation Steering, Sparse Interventions, Medical Visual Question Answering, Vision-Language Models, Decision Preservation, Calibration, Risk-Aware Adaptation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_CgffOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces a new method of improving model's predictions on unseen data while leaving the model's weights frozen. The presented framework (called PCS-AC) adds low-rank learnable corrections to the parameters of each layer of a model at inference; the framework bounds the space of allowed corrections given the structure of the hidden states, determines the set of actions to improve the answer margin within the bounded space and learns the optimal correction values based on a training subset of the new data. The authors study the application of PCS-AC to vision-language models (VLM) and text models for improving the binary predictions on medical visual-question-answering (VQA) datasets and text binary assertion tasks. On all of the considered datasets the framework shows an improvement in prediction accuracy w.r.t. the native backbone for all of the considered models and in many cases the framework outperforms other corrective intervention methods, such as LoReFT.

Critical strengths

To the best of my knowledge, the framework introduced in this paper is a novel contribution to this field of research. The authors extensively explain the details of their framework and demonstrate how it relates to other methods and approaches existing in the literature. The authors clearly demonstrate on a given set of models and datasets that their method substantially improves the accuracy for the specified set of tasks and applications, hence the authors achieve the stated goal of their research. The authors also perform a substantial number of experiments (hyperparameter scans, ablation, etc.) to support their conclusions.

Critical weaknesses

I did not identify any critical weaknesses in the paper, but I have a few concerns whose resolution would further improve it. The authors claim that simply fine-tuning the models on an unseen data might introduce unnecessary errors, but they don't show what level of accuracy can be actually reached with fine-tuning and how it compares to their method in terms of accuracy and computational costs. It would be nice to see more data and information regarding the trade-off between accuracy and e.g. computational time for fine-tuning and PCS-AC. In addition, the authors considered two VLM models, but only one text model - it would be interesting to see how the proposed framework performs on at least another text model.

Questions for authors

My questions to the authors directly follow from the matters I discuss in "Critical Weaknesses" section.

  1. Would it be possible (time-wise and resource-wise) to add a comparison between fine-tuning and PCS-AC in terms of accuracy and computational costs at least for one dataset and one type of models?
  2. Would it be possible to consider an additional text model?
Reviewer_KEVYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PCS-AC, which learns input-dependent hidden-state interventions to improve a frozen model’s binary predictions. It uses answer-margin gradients to construct an output-sensitive subspace, fits a low-rank controller to label-guided corrective actions, and then refines the controller through the frozen backbone. Experiments cover two vision–language models and one text model, with no labels, gradients, or additional backbone forward pass required at inference.

Critical strengths
  1. Strong same-domain results with informative controls. PCS-AC outperforms threshold-calibrated Native, and J-PCA exceeds random and activation-PCA subspaces across all seven InternVL environments. Balanced accuracy and repair/damage counts provide useful detail beyond overall accuracy.

  2. The closed-form compilation stage has practical value. On Qwen, compilation alone reaches 80.38%, close to the full method’s 80.95%, showing that most of the improvement is available without iterative refinement (Table 4b).

  3. The mathematical construction is clearly explained. The minimum-norm teacher action under local linearization and the reduced-rank ridge solution have explicit derivations that connect the objectives to the algorithm.

Critical weaknesses
  1. The advantage over a supervised classifier on frozen hidden states remains unclear. Figures 3–4 include an “independent logistic readout,” but its training procedure and overall performance are not reported. In this binary-prediction setting, the missing comparison makes it difficult to assess the additional benefit of intervening inside the model.

  2. The additional benefit of compilation after refinement is not well established. Full PCS-AC improves over zero-output initialization by approximately 1.8 and 1.1 percentage points on the two VLMs, averaged over two seeds. The text control retains the compiled input map and outperforms the full method, so it does not test performance without compilation (Table 4b).

  3. The text comparison uses unequal training and tuning budgets. PCS-AC receives development-set configuration selection and eight refinement epochs, while LoReFT uses a fixed configuration and two epochs (Table 14). These differences make it difficult to attribute the performance gap to the methods themselves.

Questions for authors
  1. Could the authors clarify how the independent logistic readout in Figures 3–4 was trained, and report its accuracy and balanced accuracy on the full 2,100-question holdout?

  2. For the text experiment, was a control without the compiled input map evaluated? The reported zero-output variant retains this map, so it does not fully separate compilation from subsequent supervised training.

Reviewer_VEcROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PCS-AC, a framework for correcting frozen models through input-dependent hidden-state interventions. A budget-weighted Jacobian identifies an output-sensitive subspace, within which source labels define bounded corrective actions. Reduced-rank regression compiles these actions into a conditional controller, followed by refinement through the frozen backbone with a margin-preservation penalty. Evaluation covers binary radiologic questions and synthetic text assertions.

Critical strengths
  1. The separation between identifying output-sensitive directions, constructing corrective targets, and predicting those targets is well motivated.

  2. Useful accounting of deployment cost and correction outcomes. The paper reports repairs and newly introduced errors separately, distinguishes stored assets from trainable parameters, and measures paired inference overhead.

Critical weaknesses
  1. My main concern is whether the proposed intervention offers an advantage over training a classifier on the same frozen representations. Both tasks are evaluated as binary classification, so a logistic-regression or linear-probe baseline trained on the same source data is directly relevant. Appendix O includes an independent logistic readout in selected examples, but does not report its aggregate performance. Without this comparison, it is difficult to assess whether modifying the hidden states is preferable to learning a new output classifier.

  2. The evaluation split does not establish patient-level generalization. Appendix D excludes shared identifiers and exact duplicates, but explicitly states that patient independence has not been established. These checks do not rule out different images from the same patient appearing across splits. This does not demonstrate leakage, but it leaves uncertainty about what the reported gains generalize to. The text evaluation has a related limitation: its subsets were examined during earlier development. A patient-disjoint visual evaluation and a previously unused text test set would provide stronger evidence.

  3. The contribution of action compilation is not yet convincing. Table 4(b) shows that zero-output initialization followed by refinement reaches 79.17% on Qwen and 80.55% on InternVL, compared with 80.95% and 81.62% for PCS-AC. These results suggest that J-PCA and supervised refinement account for much of the final performance. The compiled-only Qwen result is encouraging, but the paper should establish whether compilation improves convergence, final accuracy, or both. Learning curves and uncertainty estimates would help interpret these relatively small differences. For text, zero-output initialization performs better, although that control retains the compiled input map.

The preservation results provide limited support for the motivation. \textit{Keep} reduces newly introduced errors, but also reduces successful corrections. Appendix L does not establish an advantage over simply lowering intervention strength. Thus, the results show more conservative correction, rather than a clearly better correction–preservation trade-off. The transfer results are also mixed: both models lose accuracy on VQA-RAD, and Qwen drops from 80.00% to 63.04% on MRI hip. These findings limit the evidence that the controller can preserve correct predictions outside the source distribution. 35301_PCS_AC_Compiling_Output_

Questions for authors
  1. Can the authors provide a budget-matched text comparison with LoReFT? PCS-AC uses eight refinement epochs and a development search, whereas LoReFT uses a fixed two-epoch configuration. The current comparison does not isolate the effect of the proposed method.

  2. How often do the teacher actions in Equation 4 improve the actual nonlinear model margin as intended? Comparing predicted and realized margin changes would help assess the quality of the corrective targets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.