acceptodds
Under review as a conference paper at ICLR 2027

Free-to-Answer: Rethinking Output-Format Constraints in Reinforcement Fine-Tuning for Fine-Grained Visual Recognition

Abstract

Visual concepts in the real world are naturally organized from coarse semantic groups to increasingly fine-grained sub-categories, whose recognition requires both rich category knowledge and sensitivity to subtle visual differences. Rule-based reinforcement fine-tuning (RFT) has recently improved the fine-grained visual recognition (FGVR) capabilities of Multimodal Large Language Models (MLLMs). To enable automatic reward computation, existing methods commonly require responses to follow rigid output structures, such as separate reasoning and answer tags. However, it remains unclear whether these structures are merely parsing interfaces or whether they also affect recognition behavior. In this work, we systematically investigate how output-format constraints affect FGVR before and during reinforcement fine-tuning. Across five benchmarks, removing mandatory structural tags generally improves recognition accuracy, although the effect varies across models and datasets. Motivated by this observation, we develop Free-to-Answer, which optimizes free-form responses using LLM-based label extraction followed by deterministic verification. Under Open-Ended QA evaluation, Free-to-Answer achieves higher average accuracy than format-constrained RFT across the 1-, 2-, and 4-shot settings, while controlled experiments suggest that the improvement is not solely attributable to the reward design. We further analyze how free-form responses interact with closed-set label grounding. Our results reveal that rigid output structures can affect recognition behavior in visual RFT, rather than serving only as interfaces for reward parsing.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Fine-Grained Visual Recognition, Multimodal Large Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_waz4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper study and investigate into how output-format constraints affect fine-grained visual recognition (FGVR) before and during reinforcement fine-tuning. They find that across five benchmarks, removing mandatory structural tags generally improves recognition accuracy, although the effect varies across models and datasets.

Critical strengths
  1. The writing is clear
  2. They find that across five benchmarks, removing mandatory structural tags generally improves recognition accuracy, although the effect varies across models and datasets.
  3. The qualitative examples are interesting.
Critical weaknesses
  1. The studied question is very narrow-domain and niche and I don't think this would relate to ICLR.
  2. The eval benchmarks are limited as well. Even on such narrowed domain, I know there 11 datasets used for classification tasks not just 5.
  3. The method does not consistently outperform previous approach ViRFT.
Questions for authors

N/A

Reviewer_wqeZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes free-to-answer, a format free training approach for fine-grained image recognition tasks. Free-to-answer removes the standard tag based formatting used during CoT based RL post-training and instead uses a LLM based reward from the free-from response. The proposed method shows improvement on both open-set and closed set setting across five fine-grained image classification datasets.

Critical strengths

None

Critical weaknesses
  1. The core message of the paper is that using formatting during generation via tags hurts the recognition accuracy. However, this core message is not validated precisely. When the inference is changed from format to free-form, the judge is also changed from string match to LLM (From the paper, it’s not clear if the same LLM judge is used for format baseline evaluation as well or not). Therefore two things are changing at the same time and it’s unclear whether the gains come from removing the format or using a LLM as a judge which is better able to match predictions against the ground-truth class.
  2. While works such as CLS-RL and DiVE-k are discussed in related work but are not compared in the results section.
Questions for authors
  1. MCQ setting and Closed set setting are used interchangeably and it’s confusing. Could the author make it consistent? If they refer to different things please clarify
  2. Can the author provide some insights into why performance always degrades for PET dataset across all training based settings?
  3. Same as weakness 1. For format answer baseline, how is the answer evaluated? using string match or LLM? if string matches then the gain can be purely from LLM capability to match categories better. must consider using LLM even for format cases.
  4. Why no comparison is made to format the answer baseline in table 5?
  5. Sec 4.6, Isolating structural format constraints, the experiment setup is not clear. line# 458-459, why is the format reward removed if it retains the structured prompt?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.