acceptodds
Withdrawn from ICLR 2027

ROUTE, FUSE, AND VERIFY: COUNTERFACTUAL EVIDENCE-GUIDEDEXPERTROUTINGFORWHOLE BODYHUMANRECOGNITION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal human recognition can exploit complementary face, gait, and body-appearance experts, but existing methods typically adapt only one part of the recognition pipeline, such as model weighting, expert selection, or score calibration. Because these stages rely on separate signals, the system lacks a candidate-specific representation indicating whether the selected models provide sufficient identity evidence and whether the query may come from outside the gallery. As a result, they lack a direct way to determine whether the current evidence is sufficient for an identity decision or whether an additional expert is likely to provide useful identity information. We propose a query-adaptive framework that connects model routing, score fusion, and open-set recognition through calibrated identity evidence. For each selected model subset, the Calibrated Identity Evidence Model (CIEM) estimates candidate genuineness and query unknownness, using their difference as a verification certificate. The certificate guides a multimodal policy optimized with GRPO to select an anchor expert, invoke complementary experts, or stop inference. Its routing reward combines retrieval utility, verification and open-set safety, decision correctness, and the cost of additional expert calls. Given the selected route, Anchor-Preserving Temporal-Consensus (APTC) uses the anchor's complete score vector as the global base and adds high-confidence temporal evidence from complementary experts. Certificate-Reanchored Identity Scoring (CRIS) then projects CIEM certificate margins onto the APTC identity ranking through an order-preserving isotonic transformation. Experiments on CCVID, MEVID, and LTCC show that the resulting pipeline improves retrieval, verification, and open-set identification over fixed fusion, quality-guided fusion, and metric-reward routing baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Biometric, LLM, RL, Multimodal

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_brL2OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a whole-body human recognition framework that introduces CIEM to estimate query unknownness and APTC to fuse score vectors from adaptively selected recognition models. A VLM trained with GRPO serves as the router, guided by the proposed APTC-aligned route reward to determine which experts to invoke for each query.

Critical strengths

The proposed CIEM provides a candidate- and expert-subset-specific evidence signal that is consistently used to guide GRPO-based expert routing and open-set decision making, while APTC and CRIS further connect adaptive evidence acquisition with score fusion and calibration.

Critical weaknesses
  1. The paper quality is poor. The manuscript does not make effective use of the available page budget; the layout, Fig/Tab references, and table formatting are all unprofessional.
  2. The novelty is trivial compared with FusionAgent. Much of the overall pipeline still follows the same basic paradigm of query-adaptive expert selection and score-level fusion. The introduced APTC-based fusion and the APTC-aligned routing reward seem largely incremental, and the paper does not clearly demonstrate the motivation and insight of the proposed content. Moreover, the results are also marginal.
  3. The manuscript gives the impression of being incomplete or even almost generated by AI. I don't think this quality can reach the top conference like ICLR.
Questions for authors

Based on the quality of the paper, I don't have any questions for the author.

Reviewer_SzfEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The main idea of one calibrated evidence signal shared across routing, fusion and open-set calibration looks good. Motivation and intro is fine.

Critical strengths

The paper proposes a single calibrated evidence quantity that is conditioned on both the candidate identity and the currently selected expert subset, and reuses it for sequential expert acquisition and for the final open-set decision. The overall motivation and idea is interesting.

Critical weaknesses
  1. The contribution is not clear and there are quite a few details missing. For instance, title says conterfactual but I did not find counterfactual analysis in the paper.
  2. There are a few inconsistencies within the paper. Details are given below.
Questions for authors

I would like the authors to clarify the following:

  1. In the paper, the routing policy’s contribution is not isolated. ACT under learned routing (Table 3) is behind FusionAgent on Rank-1, mAP and FNIR. Please add FusionAgent routing + APTC and CRIS, and a simple threshold policy that calls experts until the CIEM margin turns positive.

  2. ACT is not defined and it should be introduced explicitly as FusionAgent’s anchor-based confidence top-k fusion, with proper citation.

  3. CRIS looks like CIEM’s margin under a monotone constraint. A baseline using CIEM’s maximum margin directly for open-set scoring would show whether CRIS is adding anything or not.

  4. Inference cost motivates the proposed method but what is the inference cost of the proposed approach. Please add average expert calls per query and route distribution. Figure 6 includes an additional expert call after acceptance; aggregate stopping numbers are needed to assess the effect of the cost penalty.

  5. The authors should explain how disagreements are resolved and report the accuracy of the actual agent answers alongside the score-based metrics.

  6. Claims in Section 4.2 and Table 1 do not match completely. QME and AdaFace beat Rank-1 on CCVID, FusionAgent beats mAP, and FNIR intervals overlap on MEVID and LTCC.

  7. CIEM is unclear and should be expanded. Please write out the softplus loss, the conditioner, the pseudo-unknown construction and the data used to calibrate tau_R.

  8. There are several internal inconsistencies, for example: CIEM versus CIWM, 0.25 versus 0.1 answer-reward weight, 48.4 versus 84.4 TAR in Table 4.

  9. “counterfactual” in the title with no counterfactual component in the method - did I miss something?

  10. “Certificate” suggests a guarantee; the authors should state precisely what it certifies and what are the gaurantees.

  11. What does 10-shot mean? ten samples per identity, and from which split? Which components are updated: the router, energy encoder, conditioner, CIEM heads, or thresholds? Table 2 gives FusionAgent results for only one of three transfer directions.

  12. The bibliography needs a careful review. Section 2.1 cites several ICCV 2025 papers unrelated to fusion or open-set recognition, and IDSelect is cited but never compared. AdaFace pages are wrong. Further, ToolRL, Pixel Reasoner and BiggerGait are NeurIPS 2025, not 2026. Here is one example.

From the paper - “Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel space reasoning via curiosity-driven reinforcement learning. Advances in Neural Information Processing Systems, 38:8222–8251, 2026a.” is a NeurIPS 2025 paper - https://openreview.net/forum?id=VeZkY3JjWV

Reviewer_nyRGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper combines identity-evidence estimation, sequential expert routing, score fusion, and open-set calibration for multimodal human recognition.

Critical strengths
  1. use a shared evidence signal for expert selection and open-set decisions is well motivated.
  2. the experiments cover several datasets and evaluation settings. the results on MEVID are promising.
Critical weaknesses
  1. The methodological novelty is limited. The improvement over FusionAgent needs to be clearer. These changes seem incremental, although combining them is useful.
  2. It is unclear how much CIEM helps model routing. The experiments show the benefit of final calibration, but do not clearly separate this from the benefit of using CIEM to select models. 3.The results are not consistently better than FusionAgent. Although the gains on MEVID are more encouraging, the results on the other two datasets do not clearly show that the added components bring a substantial benefit.
Questions for authors

The framework is reasonable and potentially useful, but its contribution currently appears primarily incremental. I lean toward Weak Reject.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.