- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents localization under degradation as a tool-composition problem solved by an MLLM agent. The agent selects restorers, super-resolution, and specialist detectors, and arbitrates among their outputs per instance. LocAS-X adds an experience memory of per-profile tool win-rates from a few labeled images. The paper claims localization in "any scene" with "any category".
Critical strengths
Timely framing with a real observation. The paper shows that whether to restore at all is an image-dependent decision (the restored-vs-original selection; App. F shows LocAS-X skips restoration and super-resolution where the rule-based version always applied them). That is a useful empirical point for the restoration-for-recognition literature.
Critical weaknesses
- Conceptual novelty is limited and the closest work is not discussed. Tool-using visual agents (VisProg, ViperGPT, HuggingGPT, MM-REACT) and LLM-driven restoration agents (AgenticIR, RestoreAgent, MAIR, 4KAgent, which is cited but not compared) already do "perceive → choose a restoration tool → verify". LocAS combines such an agent with detector routing and a candidate-selection step, but does not compare against an AgenticIR-style restoration agent followed by a strong detector. EAC is effectively a per-profile lookup of best tools learned from 80 labeled images, i.e., a few-shot supervised policy. Calling it "experience-aware calibration" that "progressively refines its decision policy" overstates what is a static retrieval table.
- The "any scene / any category" claim is not tested. Four of the eight benchmarks have an expert purpose-built from their own domain, and every degradation type has a matching restorer. There is no held-out evaluation: no leave-one-domain-out run with the matching expert and restorer removed, no unseen degradation types (e.g., snow-only, sandstorm, infrared), and no category lists outside the experts' coverage. Without these, the results show that a well-stocked toolbox matched to the test domains works, not that the agent generalizes.
Questions for authors
- Leave-one-domain-out results: remove the matching expert (and restorer) for a dataset and re-evaluate.
- Does the LLM agent beat a non-LLM controller, e.g., a CLIP or small CNN degradation classifier plus fixed rules, using the same tools?
- How sensitive is LocAS to the choice of agent model (a weaker or stronger MLLM)?
6 comments
Our Response to W1
W1:
We appreciate the reviewer's concern regarding conceptual novelty. We would like to emphasize that our contribution is not the general concept of tool-using agents, but a localization-oriented agentic framework designed for diverse visual degradations.
A. Comparison with Existing Agentic Frameworks.
We acknowledge the relevance of existing visual and restoration agents, including VisProg, ViperGPT, AgenticIR, and 4KAgent. During our initial investigation, we attempted to build a localization pipeline based on 4KAgent, but encountered two limitations: (1) its iterative restoration process required over 10 minutes per image in our preliminary experiments, and (2) the restored images did not consistently benefit downstream localization, as restoration agents primarily focus on perceptual quality. These findings motivated our development of SAIR, which prioritizes localization-oriented restoration with lower computational overhead, and MEL, which incorporates adaptive expert detector routing and instance-level prediction refinement. Importantly, rather than simply connecting an existing restoration agent with downstream detectors, we redesign the entire agentic workflow around the localization objective, coordinating scene perception, adaptive restoration, expert selection, and instance-level reasoning into a unified decision-making framework.
B. Experience-Aware Calibration (EAC).
We agree that EAC is a lightweight retrieval-based calibration mechanism rather than a continuously evolving policy. We explored more complex alternatives, including decision-trajectory distillation, but found that directly utilizing localization outcome statistics provided more stable end-to-end performance. Unlike a fixed per-scene tool lookup, EAC retrieves similar fine-grained scene profiles and provides outcome-based recommendations at multiple decision nodes, allowing the agent to jointly consider current visual observations and historical experience. As demonstrated in Appendix F, this mechanism effectively guides tool selection and consistently improves localization performance. We have also revised potentially misleading descriptions of policy refinement (e.g., Lines 29–32 and 285–288) to clarify the actual mechanism.
Overall, our research is problem-driven, focusing on robust localization under diverse degradations. We believe the task-specific design and coordination of SAIR, MEL, and EAC offer meaningful advances over existing restoration-oriented agents by jointly addressing image enhancement, expert selection, and localization reasoning.
Our Response to W2 and Summary
W2:
We appreciate the reviewer's concern regarding the generalization capabilities of LocAS. We would like to clarify that "Any Scene / Any Category" primarily refers to the flexibility and extensibility of the agentic localization framework, rather than guaranteeing optimal performance of current LocAS-X version under every unseen condition.
1. Generalization and Extensibility of Agentic Localization.
As discussed in Fig. 1, conventional representation learning and fixed pipeline approaches typically require dataset-specific retraining to handle new degradation conditions or object distributions. In contrast, LocAS dynamically coordinates available restoration and detection tools, allowing new domain-specific experts to be incorporated without redesigning the entire localization pipeline. For instance, localization in infrared environments could be supported by simply integrating suitable infrared detectors or enhancement tools. We believe this modular adaptability is an important advantage of agentic localization in diverse and dynamic environments.
2. Additional Leave-One-Out Experiments and Cross-Dataset Generalization.
To further evaluate generalization beyond existing domain-specific tool configurations, we conduct three additional leave-one-out experiments in Appendix K. First, the Leave-One-Domain-Out experiment on DarkFace evaluates the impact of removing its corresponding expert detector and restoration tool (Appendix K-i). Second, the Leave-One-Calibration-Out experiment investigates whether EAC can transfer experience without target-dataset calibration (Appendix K-ii).
More importantly, DAWN and ExDark provide two important examples of cross-dataset generalization, as neither dataset is used for domain-specific expert fine-tuning, and both are evaluated without their corresponding EAC calibration samples in the held-out settings. DAWN contains diverse adverse weather conditions, including fog, rain, snow, and sandstorms, while ExDark covers challenging low-light environments with diverse object categories. Notably, no dedicated restoration tools are available for snow or sandstorms in current LocAS-X.
To further quantify this generalization capability, we conduct a Leave-One-Dataset-Out experiment on DAWN in the following table (Appendix K-iii). When DAWN training data are excluded, YOLOv11-Large suffers a substantial F1-score decline from 45.77 to 12.43. In contrast, LocAS maintains 47.71 without any expert detector fine-tuned on DAWN, while LocAS-X achieves 51.69 without DAWN-specific experience calibration. Together with the strong localization performance on ExDark, these findings suggest that agentic localization can generalize beyond target-dataset-specific training and calibration, rather than merely relying on experts specifically trained for the evaluated datasets.
| Model | Dataset Data | [email protected] |
|---|---|---|
| YOLOv11-Large (w/o Training Data) | × | 12.43 |
| YOLOv11-Large (w/ Training Data) | ✓ | 45.77 |
| LocAS (w/o EAC) | × | 47.71 |
| LocAS-X (EAC w/o DAWN Data) | × | 51.69 |
While we do not claim universal coverage of every possible scene or category in current version of LocAS-X, we believe these additional experiments, together with the modular extensibility of LocAS, provide meaningful support for robust and generalizable localization under diverse and dynamically changing visual conditions.
Summary
We sincerely appreciate the reviewer's thoughtful comments. We understand that the primary concerns focus on the conceptual novelty of LocAS, its generalization capabilities, and the effectiveness of agentic decision-making. We believe our clarifications and additional experiments have substantially addressed these concerns. First, our contribution lies in redesigning the entire agentic workflow specifically for localization, rather than simply combining existing restoration and detection tools. Second, our leave-one-out experiments, together with the strong results on DAWN and ExDark without dataset-specific expert fine-tuning, provide evidence of the framework's generalization beyond matched training domains. Third, our comparison with alternative decision strategies, particularly Zero-Shot CLIP, demonstrates the substantial benefits of agentic decision-making beyond simple scene classification and tool selection.
Overall, we firmly believe that the proposed framework offers a meaningful and practical direction for robust localization under diverse visual degradations. We sincerely thank the reviewer for helping us strengthen the manuscript and welcome any further questions or suggestions for additional experiments, which we would be happy to address whenever feasible.
Our Response to Q1-Q3
Dear Reviewer ky4g,
We sincerely appreciate your careful evaluation of our manuscript and the valuable suggestions provided. Taking into account the feedback from all reviewers, we have performed additional experiments, expanded the technical explanations, and refined the organization of the manuscript to enhance its clarity and completeness. For ease of reference, all revisions are highlighted in blue in the updated manuscript.
Q1:
We appreciate this suggestion and have conducted additional leave-one-domain-out experiments on DarkFace to assess the role of domain-specific restoration and detection tools. Specifically, we evaluate three configurations by removing the expert face detector, the brightness enhancement restorer, and both tools simultaneously.
As reported in the above table, the F1-score decreases from 44.87 to 41.15, 7.90, and 5.45, respectively. Notably, removing brightness enhancement leads to a substantially larger performance degradation than removing the face detector alone, suggesting that effective restoration is particularly important for downstream localization in low-light environments. These findings highlight the complementary roles of restoration and detection tools within LocAS and support the importance of their coordinated use in agentic localization. We have added the complete results and discussions to Appendix K-i) of the revised manuscript.
Q2:
A. CLIP-Based Controller.
Thank you for this valuable suggestion. To evaluate LLM-based decision-making against simpler alternatives, we introduce four strategies: Fixed, Ensembling, Random, and Zero-Shot CLIP. In particular, Zero-Shot CLIP employs scene classification with the prompts "A photo of {c} scene" and "A photo with {c} objects" to select the top-K detectors by logit similarity scores. Although this non-LLM baseline performs best among the alternatives, it still trails LocAS-X by 16.97 F1 points on average across three datasets. These results support the benefits of agentic reasoning beyond simple scene-aware tool selection. Please refer to Fig. 5(b) and Lines 504–518 for detailed comparisons.
B. CNN-Based Controller.
CNN-based degradation classifiers provide another reasonable alternative, but their reliance on predefined degradation categories may limit flexibility under unseen conditions, usually require classifier retraining. In contrast, LocAS can incorporate new restoration and detection tools. We agree that an additional CNN-based comparison would be informative and would consider including it in the camera-ready version, if feasible, or future work.
Q3:
We appreciate this important question. During our exploration, we tested Qwen3-VL-8B-Instruct and Qwen3.5-9B as alternative central agents. While both models exhibit relatively stable scene perception, their top-K detector selections occasionally differ, reflecting model-specific decision preferences. We found that these differences can be alleviated by refining the decision instructions in the system and user prompts.
More importantly, EAC further improves decision consistency by incorporating previously acquired experience into the decision-making process. Instead of relying solely on the inherent reasoning preferences of individual MLLMs, LocAS-X leverages retrieved experience to provide additional guidance for tool selection (see Appendix F). We therefore expect the experience-aware mechanism to reduce model-dependent decision variations and improve the robustness of agentic localization across different central MLLMs.