- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper tried to study how language models can map occupation codes from different institutions into a common classification system. The authors investigate whether reinforcement learning can improve performance on difficult groups of data and whether providing additional information can help. The experiments compare different training methods to evaluate their effectiveness in improving model accuracy and reliability across different data groups.
Critical strengths
The problem the authors are trying to address is important and worth studying. Improving fairness across different groups is a meaningful research direction, and I agree with the motivation behind this work.
Critical weaknesses
-
This paper is very difficult to follow. The abstract is particularly hard to understand, with too many technical terms, abbreviations, and numbers introduced without clearly explaining the main idea. The introduction does not help much either. I think the writing and overall structure need serious improvement to meet the expectations of this venue. Some parts also feel repetitive and unnecessarily complicated.
-
I found the experimental sections difficult to follow as well. The text contains too many numbers and often repeats information that is already available in the tables. I think the purpose of the text should be to explain what these results mean, what trends we can observe, and where the proposed methods perform well or poorly. The experiments would be much more informative if the authors focused on explaining the results rather than simply reporting them.
-
As I understand, in Section 3.1 authors used the test results to identify poorly performing groups and this information influenced some of the later experimental choices. The authors acknowledge this limitation, but I am concerned that the same test data was also used to evaluate the later experiments. This makes it difficult to know whether the reported improvements would hold on completely new data.
-
The paper shows that additional supervised training with supporting information can improve performance on difficult groups. However, I could not find a direct comparison where the model receives the same amount of additional training without this information. Without such a comparison, it is difficult to understand whether the improvement comes from the additional information or simply from further training.
-
I am not convinced about the practical value of the more complex RL approach. The results show that simpler supervised methods with confidence-based filtering often perform better. I think the paper should better explain what the proposed approach actually offers over alternatives.
Questions for authors
Please refer to the Weaknesses.