acceptodds
Withdrawn from ICLR 2027

LASER: LLM-Guided Severity-Aware Report Encoding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Unstructured radiology reports contain rich clinical information, yet converting them into structured representations that capture the severity of findings remains challenging. Pre-trained language models compress specialized clinical text into a small region of their feature space, causing clinical severity distinctions, such as discrete versus massive calcification, to collapse into indistinguishable representations. We propose LASER, a label-efficient framework that reshapes the latent space of a text encoder to explicitly separate clinical concepts and impose a continuous severity ordering, without any manual annotation. An LLM generates synthetic sentences for each clinical concept–severity combination, providing structured training signal at zero labeling cost. A multi-objective training scheme separates clinical concepts and builds a continuous severity ordering within each concept, amplifying distinctions compressed by generic pre-training. We evaluate LASER on two datasets across two languages and organ systems, a German cardiac CT cohort and the English CheXpert Plus chest X-ray dataset, on severity classification and report retrieval. LASER consistently outperforms the baselines on both datasets and tasks, demonstrating that LLM-guided synthetic supervision is an effective and scalable path toward structured, interpretable clinical text representations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Report encoding, Contrastive learning, Synthetic data

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tQm8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LASER, a label-free framework for learning radiology sentence embeddings that distinguish clinical concepts while preserving severity ordering. It uses LLM-generated synthetic sentences across concept–severity pairs to fine-tune multilingual E5-large with five complementary losses. At inference, concept and severity labels are assigned through k-NN classification using a synthetic codebook, while report retrieval uses cosine similarity between mean-pooled sentence embeddings. The method is evaluated on English chest X-ray and German cardiac CT reports against pretrained encoders, with experiments covering classification, retrieval, severity ordering, and loss-function ablations.

Critical strengths
  1. Strong experimental evaluation: The authors compare LASER against pretrained encoders and fine-tuned baselines using the same synthetic data, including cross-entropy classification and Rank-N-Contrast. On CheXpert, LASER800 achieves a QWK of 0.632, outperforming Classifier800 (0.528), RNC800 (0.412), and RadBERT (0.350). The evaluation also includes five random seeds, ablation studies, and datasets spanning two languages and imaging modalities.

  2. Well-motivated problem and practical approach: Learning severity-aware clinical embeddings without manual annotations is an important problem. The use of LLM-generated synthetic sentences and a k-NN codebook provides a simple, reusable approach that could generalize to other clinical findings.

Critical weaknesses
  1. Inconsistent performance across datasets: LASER shows clear improvements on CheXpert but not on the cardiac dataset, where Classifier200 achieves higher accuracy, F1, and QWK than LASER200 (Table 8), and RNC outperforms LASER in accuracy at several training sizes (Table 10).

  2. Overstated claims: The abstract and Discussion claim consistent superiority, but E5-large outperforms LASER on several cardiac classification and retrieval metrics (Tables 1–3, 12). Additionally, removing the concept loss yields comparable or better severity performance (Tables 6–7).

  3. Potential test-set bias: The authors select the number of synthetic training sentences based on test-set performance without describing a separate validation split, potentially biasing the reported results.

  4. The evaluation lacks supervised models trained on the available labeled data, zero-shot LLM grading, and simple keyword-based baselines, making it difficult to assess LASER's practical advantages.

  5. Performance varies with the number of synthetic sentences, without sufficient explanation. Moreover, the correlation values reported in Section 5.3 do not match Table 5, raising concerns about reporting accuracy.

Questions for authors
  1. Was any real labeled data used for hyperparameter selection, including the choice of synthetic training size (200/800)? If not, what validation procedure was used?

  2. Why do the severity correlation values reported in Section 5.3 differ in sign and magnitude from those in Table 5? Which values are correct?

  3. Were the Classifier and RNC baselines independently tuned, or did they use LASER's hyperparameters? How does LASER compare against supervised and zero-shot LLM grading baselines?

  4. Why does performance decline as the number of synthetic training sentences increases? Could limited diversity or duplication in the generated data explain this trend?

Reviewer_6zR4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

LASER studies concept and severity encoding in radiology reports. The paper use LLM to generates synthetic sentences for predefined concept–severity combinations with four severity grades. And use these data to finetune a E5-large with five objectives: concept classification, same-concept/same-grade contrastive learning, different-grade separation, triplet loss, and cosine regression. Experiments is conduct on German cardiac CT reports and English CheXpert Plus reports.

Critical strengths
  1. Preserving attributes of concepts in radiology reports like severity has practical utility. And it is important for report encoder to encode these information in Medical AI applications. However, focusing solely on severity is far from sufficient.
Critical weaknesses
  1. Severity is not the only import attributes that is critial in radiology reports. For example, the size or the location of a lesion is also important. However, this paper only focus on a very narrow point on this topic, and fail to explore a more borad question: how to encode medical concept and its diverse attributes to medical text embedding.
  2. The methodological contribution is insufficiently distinguished from an established combination of synthetic supervision and metric learning.
Questions for authors

See weakness.

Reviewer_hnGQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes LASER, a method for automatically identifying clinical findings and their severity levels in radiology reports without manually labeled training data. This method use LLM to generate example sentences describing each type of finding at different severity levels. These synthetic sentences are used to fine-tune the multilingual E5-large model with five training objectives that help it distinguish different findings and organize their severity levels in order. The model assigns severity labels to new report sentences by comparing them with similar labeled synthetic sentences. It also combines sentence information to retrieve similar reports. The method is evaluated on 700 English chest X-ray reports from CheXpert Plus and 130 internal German cardiac CT reports, where it generally outperforms pretrained text encoders on severity classification and report retrieval.

Critical strengths
  1. A real problem with a label-free solution. Pretrained text encoders usually are not trained to encode finding severity, which is important to clinical use of radiology reports. Using LLM-generated sentences as labeled anchors avoids human annotation and makes predictions traceable to example sentences, which gives the approach practical value for clinical NLP.

  2. Evidence of synthetic-to-real transfer. The training sentences are generated from concept and severity keywords alone, without access to any real reports. Models trained only on these sentences still improve severity classification on real English chest X-ray and German cardiac CT reports over off-the-shelf encoders. This suggests that synthetic supervision can transfer to real clinical text, which matters for clinical settings where labeled or shareable report data is scarce.

Critical weaknesses
  1. Key baselines are missing. 1.1: The method relies on an LLM (Claude, Sec. 4.1) to generate training sentences, but the paper does not compare against using an LLM (e.g., a locally deployable open-weight model) to grade the severity of the real sentences directly. 1.2: Although the method is described as “label-efficient,” it is also not compared with a model trained on a small number of real labeled sentences (e.g., via cross-validation on the existing annotated sentences). Without these baselines, it is unclear whether LASER is preferable in practice.

  2. The benefit of the full five-loss objective is not consistently established. 2.1: On CheXpert, LASER outperforms a cross-entropy classifier and RNC trained on the same synthetic data when at least 400 sentences per concept are used (Tables 9, 11). On the cardiac dataset, however, the cross-entropy classifier beats the best LASER configuration on accuracy, F1, and QWK (Table 8). 2.2: Most ablation differences are within seed variance. Removing the concept loss even slightly improves severity QWK on both datasets, though it lowers concept accuracy on the cardiac dataset (Tables 6–7). The objective helps in some settings, but the need for all five losses over simpler training is not clearly shown.

Questions for authors
  1. Test annotations: The real test sentences are described as “manually labeled” (Sec. 4.1). Who performed the annotation (e.g., radiologists), how many annotators were involved, and what was the inter-annotator agreement?

  2. Loss design: Do the cross-entropy and RNC baselines also include the concept loss and comparable tuning? Have simpler loss combinations been evaluated under the same conditions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.