acceptodds
Withdrawn from ICLR 2027

CoralAgent: From Visual Recognition to Constrained Agentic Ecological Assessment for Coral Reef Conservation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reliable coral reef ecological analysis requires more than accurate visual recognition; it demands strict adherence to sampling protocols, taxonomic standards, and precise measurement definitions. However, existing vision-language models and general-purpose agents lack explicit mechanisms to enforce these domain-specific requirements, a challenge compounded by the scarcity of reliable coral-specific models. To bridge this gap, we introduce CoralAgent, a multi-agent framework that translates ecological requirements into explicit rules for evidence collection, computation, and validation. Its core component, Coral Structured TAsk Representation (C-STAR), formalizes measurement objectives, evidence requirements, validation criteria, recovery policies, and admissible conclusions. Guided by C-STAR, CoralAgent coordinates reusable analytical primitives, validates intermediate results, and switches to workflows adapted from conventional coral surveys when coral-specific models are unavailable or their outputs fail validation. To evaluate quantitative ecological analysis beyond recognition and segmentation, we introduce Coral-Bench, comprising 1,986 question–answer pairs and 2,535 images across 23 tasks. These tasks cover coral recognition, visual knowledge, cover estimation, community composition and structure analysis, and ecological index computation, including cross-site and cross-year analysis. Experiments with seven multimodal baselines show that CoralAgent outperforms the GPT-5.4 baseline on 22 of 23 tasks and achieves the best or tied-best performance on 20. Compared with GPT-5.4, CoralAgent improves task accuracy for single-image coral cover estimation from 20.00% to 72.59% and for Gini–Simpson diversity calculation from 19.18% to 61.64%.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Coral Reef Ecological Analysis, Agent, Benchmark, Dataset

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XuNoOpenReview ↗
Rating
3
Confidence
1
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper’s main contributions is (according to the authors) the first agentic framework for coral reef ecological analysis. Additionally, Coral-Bench is introduced, a multiple-choice question-answer for visual understanding across 23 tasks. This dataset is used to evaluate the CoralAgent framework. The benchmark contribution is rather limited, consisting of only 1.9k manually verified questions over ~2.5k, compared to for instance CoralVQA, covering 12.8k images. The benchmark appears to suggest improved task accuracy over GPT baseline.

Critical strengths

It is encouraging to see agentic AI applied to an ecological domain, as much of the existing research focuses on more conventional general-purpose task automation. However, it would be valuable to expand on the generalizability of the proposed framework to other ecological tasks (e.g., bird monitoring [1]) to better understand its broader applicability and limitations. The literature review provides limited coverage of existing agentic AI solutions addressing similar tasks (i.e., RS-Agent and Earth-Agent). A more comprehensive discussion of related agentic architectures, their evaluation methodologies, and reported limitations would help better position the proposed approach within existing research and clarify its novelty and contributions.

  1. Kryklyvets, Yevheniia, et al. "MAviS: A multimodal conversational assistant for avian species." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
Critical weaknesses

My main concern is the evaluation. Currently the benchmark is conducted only once per question; recent work [1,2,3] rise importance of VLM assessment in terms of reliability and response consistency. This is particularly problematic for multiple-choice questions because a correct answer does not necessarily demonstrate reliable reasoning or understanding. Table 2 (Appendix A.1.3) shows that a substantial portion of the benchmark relies on a one-image, one-question setup. Also, the benchmark reports accuracy per task (e.g., coral cover task has 135 questions while bleached coral recognition has 300 questions); and a single correct response does not necessarily indicate reliable visual understanding, particularly in multiple-choice settings where guessing is possible. Consider other metrics like Pass@K, paraphrase consistency, option-order robustness.

  1. Gu, Yu, et al. "The illusion of readiness in health AI." arXiv preprint arXiv:2509.18234 (2025).
  2. Sun, Yingying, Henry Levesque, and Lu Lin. "Stability Without Sensitivity: Evaluative Description and Directional Judgment in Vision–Language Models." Design and Artificial Intelligence (2026): 100082.
  3. Cheng, Zerui, et al. "Position: Benchmarking is Broken-Don't Let AI be its Own Judge." Advances in Neural Information Processing Systems 38 (2026).
Questions for authors

1.Do the authors have any insight into how stable are the reported results across repeated independent evaluations of the same image–question pairs? I am curious if the conclusions remain consistent across multiple runs.

2.How sensitive are model predictions to changes in question phrasing or the ordering of multiple-choice options? I encourage the authors to consider this evaluation.

3.To what extent do correct predictions reflect actual visual understanding rather than guessing or reliance on linguistic priors? Could an image-blind baseline be established?

4.Given the varying number of questions per task, are the observed performance differences statistically significant? Could you report confidence intervals?

5.Why is accuracy the primary evaluation metric? Could you provide additional metrics?

3 comments

W1: Reliability and response consistency of the evaluation

My main concern is the evaluation. Currently the benchmark is conducted only once per question; recent work [1,2,3] rise importance of VLM assessment in terms of reliability and response consistency. This is particularly problematic for multiple-choice questions because a correct answer does not necessarily demonstrate reliable reasoning or understanding. Table 2 (Appendix A.1.3) shows that a substantial portion of the benchmark relies on a one-image, one-question setup. Also, the benchmark reports accuracy per task (e.g., coral cover task has 135 questions while bleached coral recognition has 300 questions); and a single correct response does not necessarily indicate reliable visual understanding, particularly in multiple-choice settings where guessing is possible. Consider other metrics like Pass@K, paraphrase consistency, option-order robustness. Gu, Yu, et al. "The illusion of readiness in health AI." arXiv preprint arXiv:2509.18234 (2025). Sun, Yingying, Henry Levesque, and Lu Lin. "Stability Without Sensitivity: Evaluative Description and Directional Judgment in Vision–Language Models." Design and Artificial Intelligence (2026): 100082. Cheng, Zerui, et al. "Position: Benchmarking is Broken-Don't Let AI be its Own Judge." Advances in Neural Information Processing Systems 38 (2026).

Response:

We thank the reviewer for raising the importance of reliability and response consistency, and we agree that a single correct multiple-choice answer does not by itself demonstrate reliable understanding. We address this in three ways.

First, we added analyses in our responses below: repeated independent runs (Q1), option-order robustness (Q2), an image-blind baseline (Q3), confidence intervals (Q4) and numerical error metrics (Q5).

Second, CoralAgent’s answers do not rely on option selection alone. Each answer is derived from recorded intermediate evidence (segmentation masks, point-level labels and the resulting computation), so its correctness can be inspected beyond the final choice. Its numerical estimates also show lower error than GPT-5.4 (Q5).

Third, regarding the one-image, one-question setup, many quantitative questions require multiple dependent steps on the same image (e.g., identifying genera, computing composition, then deriving a diversity index), so a correct answer requires consistent analysis across these steps rather than a single guess.

In the revised version, we will report Pass@K and paraphrase consistency for all tasks, repeated-run results for the baselines, and discuss the works the reviewer cited [1,2,3].

Q1: Stability across repeated runs

Do the authors have any insight into how stable are the reported results across repeated independent evaluations of the same image–question pairs? I am curious if the conclusions remain consistent across multiple runs.

Response:

We thank the reviewer for this question. We evaluated CoralAgent three times on the same image–question pairs for four tasks:

Table Q1: Accuracy (%) of CoralAgent across three independent runs. Mean ± SD is computed over the three runs.

TaskRun 1Run 2Run 3Mean ± SDBest baseline
Shannon diversity index52.0553.4252.0552.51 ± 0.7928.77 (Kimi)
Pielou’s evenness index60.2761.6460.2760.73 ± 0.7949.32 (GPT)
Bleached coral cover estimation56.2557.8156.2556.77 ± 0.9031.25 (GLM/Qwen/Gemini)
Coral richness assessment55.5656.7956.7956.38 ± 0.7140.74 (Grok)

The results are stable: across runs, accuracy varies by at most one question per task (SD ≤ 0.90). The conclusions also remain consistent, since even the lowest of the three runs exceeds the best baseline on every task by 10.95 to 25.00 points. We will report repeated-run results for all tasks in the revised version.

Q2: Sensitivity to question phrasing and option order

How sensitive are model predictions to changes in question phrasing or the ordering of multiple-choice options? I encourage the authors to consider this evaluation.

Response:

We thank the reviewer for this suggestion.

Option ordering. By design, CoralAgent is insensitive to option ordering on quantitative tasks. For example, on Pielou’s evenness index calculation, CoralAgent first computes the index value from the sampled evidence and then selects the option closest to that value. To verify this empirically, we randomly shuffled the option order four times on three tasks:

Table Q2: Accuracy (%) of CoralAgent under four random shuffles of the answer options. Mean ± SD is computed over the four shuffles.

TaskShuffle 1Shuffle 2Shuffle 3Shuffle 4Mean ± SD
Pielou’s evenness index60.2760.2761.6460.2760.61 ± 0.68
Coral richness assessment55.5655.5656.7956.7956.18 ± 0.71
Bleached coral cover estimation56.2557.8156.2556.2556.64 ± 0.78

Across shuffles, accuracy varies by at most one question per task, which is consistent with the variation observed across repeated runs.

Question phrasing. We agree this is a valuable evaluation. We will conduct experiments with paraphrased questions to assess the sensitivity of CoralAgent and the baselines to question phrasing, and we will report these results in the revised version.

Q3: Image-blind baseline

To what extent do correct predictions reflect actual visual understanding rather than guessing or reliance on linguistic priors? Could an image-blind baseline be established?

Response:

We thank the reviewer for this suggestion. We established an image-blind baseline in which GPT-5.4 receives only the question and options, without images:

Table Q3: Accuracy (%) of the image-blind GPT-5.4 baseline (question and options only, no images), GPT-5.4 with images, and CoralAgent.

TaskImage-blind GPT-5.4GPT-5.4 (with images)CoralAgent
Coral cover estimation14.0720.0072.59
Family-level coral recognition26.5368.3780.61
Genus-level coral recognition22.0742.3461.71
Species-level coral recognition32.0052.0060.00
Coral composition analysis14.8127.1643.21
Dominant genus recognition34.5756.7965.43
Shannon diversity index12.3323.2952.05

Three observations follow:

(1) Linguistic priors provide little advantage. Image-blind accuracy is close to or below chance level (25%) on most tasks. Although GPT-5.4 was used to generate the distractors, it cannot recover the correct answers from the question and option text alone. The slightly higher scores on species-level recognition and dominant genus recognition likely reflect prior knowledge of common coral taxa, such as genera that frequently dominate reef communities. (2) CoralAgent’s correct predictions depend on visual evidence. CoralAgent exceeds the image-blind baseline by 28.0–58.5 points on every task. Its answers are also traceable to visual evidence, since each prediction is derived from recorded masks and point-level labels. (3) Direct prompting makes limited use of visual information on quantitative tasks. With images, GPT-5.4 improves only modestly over the image-blind baseline on coral cover (+5.9) and Shannon diversity (+11.0). This indicates that direct prompting fails to extract quantitative information from images, which is the limitation that CoralAgent addresses.

We will report image-blind results for all tasks in the revised version.

Q4: Statistical significance and confidence intervals

Given the varying number of questions per task, are the observed performance differences statistically significant? Could you report confidence intervals?

Response:

We thank the reviewer for this suggestion. We report the accuracy of GPT-5.4 and CoralAgent with 95% Wilson confidence intervals:

Table Q4: Accuracy (%) of GPT-5.4 and CoralAgent with 95% Wilson confidence intervals.

TaskGPT [95% CI]CoralAgent [95% CI]
Coral cover estimation20.00 [14.12, 27.54]72.59 [64.52, 79.41]
Bleached coral cover estimation14.06 [7.58, 24.62]56.25 [44.09, 67.71]
Bleached coral recognition39.33 [33.97, 44.96]59.00 [53.35, 64.42]
Genus-level coral recognition42.34 [36.03, 48.92]61.71 [55.17, 67.86]
Coral richness assessment19.75 [12.54, 29.70]55.56 [44.73, 65.88]
Shannon diversity index23.29 [15.08, 34.17]52.05 [40.78, 63.12]
Simpson diversity index19.18 [11.78, 29.66]61.64 [50.17, 71.95]

On all seven tasks, the 95% confidence intervals of CoralAgent and GPT-5.4 do not overlap: CoralAgent’s lower bound exceeds GPT-5.4’s upper bound in every case, including tasks with as few as 64 questions. Non-overlapping 95% intervals imply that the differences are statistically significant.

Q5: Choice of evaluation metric

Why is accuracy the primary evaluation metric? Could you provide additional metrics?

We thank the reviewer for this question.

Why accuracy. Coral-Bench covers 23 heterogeneous tasks. Their answers include taxonomic labels, numerical values, site or year labels, directional judgments and percentage-point differences. Casting all tasks in a multiple-choice format, and scoring them by accuracy, provides a single metric that is comparable across tasks and applies equally to all methods, including VLMs that cannot reliably produce well-formed numerical outputs. For quantitative tasks, the options were designed by marine scientists around the reference values, so that selecting the correct option requires an estimate within an ecologically meaningful range.

Additional metrics. For quantitative tasks, we additionally compare the numerical estimates with the expert-derived reference values. CoralAgent consistently reduces numerical error relative to GPT-5.4:

Table Q5: Coverage and numerical error (MAE/RMSE) of CoralAgent and GPT-5.4 relative to expert-derived reference values.

TaskCoralAgent MAECoralAgent RMSEGPT MAEGPT RMSE
Simpson diversity index0.1520.2090.1950.245
Pielou’s evenness index0.1600.2420.2820.361
Bleached coral cover estimation0.1170.1530.1400.176
Coral richness assessment0.6500.9751.7412.169

CoralAgent reduces MAE by 16–63% across the four tasks, with the largest reductions on coral richness (1.741 → 0.650 genera, i.e., an average error below one genus) and Pielou’s evenness (0.282 → 0.160). In the revised version, we will report MAE and RMSE for all quantitative tasks and macro-F1 for recognition tasks.

Reviewer_pxCyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents CoralAgent, a GPT-5.4-based multi-agent framework for coral reef image analysis whose core is C-STAR, a seven-field task representation (scope, invariants, evidence, derivation, validation, recovery, admissible conclusions). The implementation combines visual tools, point counting, 130 primitives, and 153 expert rules. It also introduces Coral-Bench: 1,986 multiple-choice questions over 23 tasks with expert point-count targets. Experiments report large gains over seven tool-free VLMs, a same-backbone/same-tool GPT-Agent comparison on eight tasks, and validation/recovery ablations on Gini–Simpson diversity.

Critical strengths
  1. The ecological measurement definitions are precise. Bleached coral remains live coral, cover uses the valid-benthos denominator, and unresolved genus labels are excluded from composition and diversity estimates. Coral-Bench adds 1,986 questions with point-count references, cross-annotator review, and expert auditing.

  2. Tables 9–10 provide useful controls: GPT-Agent shares the backbone and tools, while the ablations test validation and recovery. These comparisons show benefits from the full workflow beyond tool access alone.

Critical weaknesses
  1. The experiments do not isolate C-STAR's contribution. Table 9 covers eight of 23 tasks without establishing equal evidence or retry budgets; GPT-Agent also trails tool-free GPT on five of those eight. The headline single-image cover gain has no tool-using comparison. Table 10 tests only one 73-question task and removes neither C-STAR nor the expert rules from CoralAgent. A fixed pipeline with the same tools, rules, checks, and budget is needed to attribute the gains to the representation.

  2. The reliability claim needs evidence beyond multiple-choice accuracy. The paper does not report numerical cover/index errors, validation false-accept/false-reject rates, or error rates and coverage for accepted reports. Recovery success and cost are also unreported. The checks assess "operational consistency" (Appendix A.2.1), which does not establish that accepted estimates are correct. Repeated-run uncertainty is missing too; on the 20-question tasks, a single answer changes accuracy by five percentage points.

  3. The benchmark provides limited support for claims across sites and years. Those tasks contain only 180 questions (9.1% of Coral-Bench), drawn from six Moorea sites and three Flower Garden Banks sites. Although annotations undergo review, the paper gives a 95% agreement threshold without reporting the agreement actually achieved. The reliability of the reference labels is therefore not quantified.

  4. The inference-time evidence is not described clearly enough to reproduce the comparisons. It is unclear which masks and annotations are model-generated or human-provided, how they are separated from the 100-point references, and whether GPT-Agent receives the same artifacts. The fallback sample size, Compose/Compile procedures, GPT-5.4 snapshot, and calibration parameters Θ also need to be specified.

Questions for authors
  1. Which inference-time masks and annotations are model predictions, and which are human-provided? Please clarify their separation from the 100-point references, whether GPT-Agent receives identical artifacts, and the fallback point-count sample size.

  2. Do the gains persist without C-STAR when the primitives, expert rules, checks, and fallback are kept fixed? This comparison needs matched point-sampling, tool/model-call, retry, and cost budgets. Please specify those budgets for Tables 9–10, along with the prompts, tool descriptions, GPT-5.4 snapshot, and Θ values.

  3. How accurate are the numerical cover/index estimates among accepted and flagged reports, and what fraction of reports is accepted? Please report validation false-accept/false-reject rates, explain how unresolved cases enter the accuracy score, and assess sensitivity to repeated runs, reference-point sampling, and unresolved genus labels.

9 comments

General Response: Extended Ablation

We extended the full ablation to 11 tasks spanning cover estimation, recognition, community composition and diversity:

Table W1: Extended ablation on 11 Coral-Bench tasks (accuracy, %). All settings use the GPT-5.4 backbone, and all agent variants have access to the same tools.

TaskGPT (no tools)GPT-Agent + toolsGPT-Agent + tools + expert rulesCoralAgent w/o recoveryCoralAgent w/o validationCoralAgent
Coral cover estimation20.0054.0754.0771.8569.6372.59
Coral composition analysis27.1625.9320.9928.4029.6343.21
Bleached coral cover estimation14.0614.0637.5053.1348.4456.25
Coral richness assessment19.7525.9325.9351.8525.9355.56
Genus-level coral recognition42.3442.3447.7558.5658.5661.71
Family-level coral recognition68.3768.3782.6572.4578.5780.61
Dominant genus recognition56.7920.9920.9949.3858.0265.43
Shannon diversity index23.2919.1836.9943.8434.2552.05
Simpson diversity index19.1827.4027.4038.3649.3261.64
Species-level coral recognition52.0052.0053.3358.0058.0060.00
Pielou’s evenness index49.3234.2521.9250.6839.7361.64

These results support three findings:

  1. Tool access alone does not help.GPT-Agent fails to improve over tool-free GPT on 8 of 11 tasks, and drops sharply on dominant genus recognition (56.79 → 20.99). Successful tool calls can return unreliable outputs that propagate into downstream computations.

  2. Expert rules as context are not enough. Adding expert rules to GPT-Agent helps on some tasks but hurts on others, such as Pielou’s evenness and composition analysis. On family-level recognition, it slightly exceeds CoralAgent (82.65 vs. 80.61), likely because the rules directly support taxonomic identification. Overall, however, it remains well below CoralAgent, which outperforms it on 10 of 11 tasks with the same tools and expert rules.

  3. C-STAR’s validation and recovery are both necessary. Both mechanisms are specified by C-STAR, through its validation criteria (V) and recovery policy (R). Removing either one lowers accuracy on all 11 tasks.

We will include the full ablation in the main text and extend it to all 23 tasks in the revised version.

W1: Isolating C-STAR's contribution

The experiments do not isolate C-STAR's contribution. Table 9 covers eight of 23 tasks without establishing equal evidence or retry budgets; GPT-Agent also trails tool-free GPT on five of those eight. The headline single-image cover gain has no tool-using comparison. Table 10 tests only one 73-question task and removes neither C-STAR nor the expert rules from CoralAgent. A fixed pipeline with the same tools, rules, checks, and budget is needed to attribute the gains to the representation.

Response:

We thank the reviewer for this careful critique. We respond point by point.

(1) Isolating C-STAR’s contribution. C-STAR consists of three components: task semantics (Ω, I), the evidence contract (E, D, V), and recovery and reporting (R, C). Among them, validation (V) and recovery (R) form its core. Our ablations isolate it as follows:

GPT-Agent + tools + expert rules uses the same GPT-5.4 backbone, tools and expert rules as CoralAgent, but without C-STAR. It therefore approximates CoralAgent w/o C-STAR. CoralAgent w/o validation removes C-STAR’s validation criteria (V). CoralAgent w/o recovery removes C-STAR’s recovery policy (R).

(2) Evidence and retry budgets. Both methods were evaluated with the same tool set and the same limit of at most 4 retries per question. GPT-Agent never used this budget: all its tool calls executed successfully, and without validation it had no basis to question their outputs, so it accepted the first result in every case (0 retries). In contrast, CoralAgent used on average 1.738 retries per question on the single-image tasks in Table 9, triggered only when C-STAR’s validation detected insufficient evidence. The gain therefore comes not from a larger budget allowance, but from C-STAR’s ability to detect when additional evidence is needed. This is the failure mode illustrated in Figure 2b: successful execution does not imply valid evidence. We will report these budget statistics in the revised version.

(3) GPT-Agent trailing tool-free GPT. GPT-Agent relies on tool outputs, but tools sometimes return incorrect intermediate results, such as wrong genus labels or incomplete segmentation masks. Without validation and recovery, these errors propagate directly into the answer. For example, on dominant genus recognition, accuracy drops from 56.79 to 20.99. Tools do help when their outputs are reliable, as on single-image cover (20.00 → 54.07), but unchecked tool use is not consistently beneficial: GPT-Agent fails to improve over GPT on 8 of 11 tasks. The detailed results are shown in Table W1.

(4) Single-image cover lacks a tool-using comparison. We added this comparison to Table W1. With the same segmentation tools, GPT-Agent achieves 54.07%, compared with 72.59% for CoralAgent (+18.52).

(5) Task coverage (8 of 23 tasks). We extended the full ablation to 11 tasks spanning cover estimation, recognition, community composition and diversity in Table W1. We will include the full ablation in the main text and extend it to all 23 tasks in the revised version.

(6) Table 10 removes neither C-STAR nor the expert rules. As explained in (1), the extended ablation removes C-STAR’s V and R fields, and compares against GPT-Agent with the same tools and expert rules but without C-STAR. We will add CoralAgent w/o expert rules in the revised version.

(7) Fixed pipeline. We address the fixed-pipeline comparison in our response to Q2.

W2: Reliability beyond multiple-choice accuracy

The reliability claim needs evidence beyond multiple-choice accuracy. The paper does not report numerical cover/index errors, validation false-accept/false-reject rates, or error rates and coverage for accepted reports. Recovery success and cost are also unreported. The checks assess "operational consistency" (Appendix A.2.1), which does not establish that accepted estimates are correct. Repeated-run uncertainty is missing too; on the 20-question tasks, a single answer changes accuracy by five percentage points.

Response:

We thank the reviewer for this important point and report additional evidence below.

(1) Numerical errors and coverage. Beyond multiple-choice accuracy, we compare the numerical estimates with the expert-derived reference values. CoralAgent produced a numerical estimate for every question (100% coverage) and substantially reduces numerical error relative to GPT-5.4:

Table W2A: Numerical estimation error of CoralAgent and GPT-5.4 relative to expert-derived reference values. Coverage is the fraction of questions for which CoralAgent produced a numerical estimate.

TaskCoverageCoralAgent MAECoralAgent RMSEGPT MAEGPT RMSE
Simpson diversity index73/730.1520.2090.1950.245
Pielou's evenness index73/730.1600.2420.2820.361
Bleached coral coverestimation64/640.1170.1530.1400.176
Coral richness assessment81/810.6500.9751.7412.169

CoralAgent reduces MAE by 16–63% across the four tasks, with the largest reductions on coral richness (1.741 → 0.650 genera, i.e., an average error below one genus) and Pielou’s evenness (0.282 → 0.160). In the revised version, we will report MAE and RMSE for all quantitative tasks and macro-F1 for recognition tasks.

(2) Recovery success and harmful changes. We compare the initial answer with the final answer after validation and recovery. “Wrong → correct” measures recovery success, and “correct → wrong” measures harmful changes (all as fractions of total questions):

Table W2B: Effect of validation and recovery on CoralAgent’s answers (% of all questions).

TaskInitial acc.Wrong → correctCorrect → wrongFinal acc.
Family-level coral recognition73.4710.203.0680.61
Genus-level coral recognition35.5930.184.0561.71
Dominant genus recognition50.6217.282.4765.43
Simpson diversity index38.3635.6212.3361.64
Pielou’s evenness index39.7334.2512.3361.64

Recovery corrects far more answers than it harms, with net gains of 7.1–26.1 points. Harmful changes are higher on the diversity tasks (12.33%). These indices are computed from estimated genus composition, so re-sampling can sometimes shift a correct estimate across an answer boundary. We will include case studies of these analyses in the revised version. In addition, recovery cost is modest: on average 1.738 retries per question on single-image tasks, within the limit of 4.

(3) Repeated-run uncertainty. We ran CoralAgent three times on four tasks:

Table W2C: Accuracy (%) of CoralAgent across three independent runs. Mean ± SD is computed over the three runs.

TaskRun 1Run 2Run 3Mean ± SD
Shannon diversity index52.0553.4252.0552.51 ± 0.79
Pielou’s evenness index60.2761.6460.2760.73 ± 0.79
Bleached coral cover estimation56.2557.8156.2556.77 ± 0.90
Coral richness assessment55.5656.7956.7956.38 ± 0.71

Across all four tasks, the standard deviation is below 1 percentage point (0.71–0.90), and runs differ by at most one question, showing that CoralAgent’s results are highly stable across repeated runs. We will report repeated-run results for all tasks, including the 20-question tasks, in the revised version.

Reviewer_fEEFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Paper applies multimodal LLMs / VLMs as agents to identity, segment, and characterize images of coral reefs. Authors create a benchmark CoralBench to evaluate models in tasks of coral ecology - these tasks are generally visual question answering and quantitative assessments, but models likely lack such kind of domain knowledge. Authors then propose a specialized agent CoralAgent with better domain expertise about coral ecology, through context engineering and domain-specific tools. CoralAgent is shown to be better at CoralBench.

Critical strengths
  • For coral ecology, the benchmark and attaching specialized tools to general-purpose LLMs like GPT is a useful addition.
  • CoralAgent is a sophisticated orchestration of agents and their contexts, optimized to do well on CoralBench tasks.
  • CoralBench has a diverse set of questions and many LLMs are evaluated on the tasks. Dataset is open-source and will be useful for coral ecology research.
Critical weaknesses
  • The paper is extremely focused on coral reef ecology, to the extend that I struggle to understand if the agent methodology / benchmark creation can be extended to other sciences and can be useful to ML in general. The paper could be more suited for a domain science venue like a journal, or authors could explain how their approaches can be more generally applicable e.g. for the broader ecology community (soil, food etc.).
  • Figure 2 is a lot. There are two subfigures, so better to separate them and explain each part well. It'd be useful to have a running example through the methodology section, because I'm not sure what a sentence like this means "Agents coordinate through a shared state that records evidence, provenance, intermediate measurements, and validation outcomes.".
  • Figure 3 is also a lot. I don't know where and what to get out of it, other than it looks complex.
  • For writing it makes sense to motivate CoralBench first, so you can evaluate the shortcomings of simple approaches and naive prompting. Then you could address these concerns and motivate CoralAgent. Also, it feels unnecessary to present C-STAR as a contribution independent of CoralAgent, as C-STAR is just a collection of information and results about the coral images presented at evaluation.
  • There are lots of moving parts in CoralAgent. There is an ablation study on which agents and contexts are necessary for performance, and it'd be a better use of space to discuss that in the main text rather than (1) describing the methodology in such length and (2) repeating a lot of numbers of Figure 4 in Section 5.2.
Questions for authors
  • Could you present some examples of how CoralBench has more interesting questions than CoralVQA, and why you think models do not succeed at those questions?
  • The questions in appendix (figure 5..9) look quite hard, even for coral ecologists. Could you provide a human baseline on this benchmark?
  • Since the proposal is a CoralAgent, how does an agent (codex + gpt-5.4) perform on this tasks? Appendix A.3.2. does this ablation but on a small subset of tasks from CoralBench.
4 comments
AuthorsOpenReview ↗

CoralAgent: From Visual Recognition to Constrained Agentic Ecological Assessment for Coral Reef Conservation

W1: Generality beyond coral reef ecology

The paper is extremely focused on coral reef ecology, to the extend that I struggle to understand if the agent methodology / benchmark creation can be extended to other sciences and can be useful to ML in general. The paper could be more suited for a domain science venue like a journal, or authors could explain how their approaches can be more generally applicable e.g. for the broader ecology community (soil, food etc.).

Response:

We thank the reviewer for this comment. We are sorry that the paper’s coral-centered presentation did not make the generality of C-STAR clear. C-STAR is a general architecture for injecting domain knowledge in a pluggable way: its structured representation (Ω, I, E, D, V, R, C), composition grammar, rule compiler and validate–recover–revalidate loop contain no coral-specific content. Domain knowledge enters only through the primitive library and the expert-rule library, and many primitives are themselves generic, such as point sampling, area ratios, pooled aggregation and diversity indices; 82 of the 130 primitives are domain-agnostic. Transferring to a new domain therefore only requires redefining Ω and I and adding domain-specific primitives and expert rules.

For example, vegetation cover estimation from quadrat images mirrors our coral cover task. The scope (Ω) becomes live vegetation cover within the quadrat frame, and the invariants (I) specify that senescent but living plants count as live, while bare soil, litter and standing dead material remain in the denominator. The evidence, derivation, validation and reporting fields (E, D, V, C) remain unchanged: cover is still derived from pixel-area or point-count ratios and validated by their agreement. The recovery policy (R) falls back to point-intercept sampling, the standard field method in vegetation surveys. Generic primitives such as valid-area definition, pixel-area measurement, random point sampling, cover-ratio calculation and pooled aggregation, together with the map/reduce operators for multi-site and multi-year comparisons, are reused directly; only the segmentation model and the domain-specific expert rules need to be replaced. If species are identified at the sampled points, the same diversity primitives (Shannon, Gini–Simpson and Pielou) also apply without modification. The same approach extends to other ecological surveys, such as soil fauna diversity assessment from microscopy counts and mangrove field surveys.

Beyond ecology, our ablations show that unconstrained tool use can even reduce accuracy: GPT-Agent fails to improve over tool-free GPT on 8 of 11 tasks. This suggests that executable measurement constraints are a broadly relevant design principle for scientific agents. In the revised version, we will add a section on the generality of C-STAR that presents this example and its cross-domain use in detail.

AuthorsOpenReview ↗

CoralAgent: From Visual Recognition to Constrained Agentic Ecological Assessment for Coral Reef Conservation

W2 & W3 & W4: Figures, running example and paper organization

Figure 2 is a lot. There are two subfigures, so better to separate them and explain each part well. It'd be useful to have a running example through the methodology section, because I'm not sure what a sentence like this means "Agents coordinate through a shared state that records evidence, provenance, intermediate measurements, and validation outcomes.". Figure 3 is also a lot. I don't know where and what to get out of it, other than it looks complex. For writing it makes sense to motivate CoralBench first, so you can evaluate the shortcomings of simple approaches and naive prompting. Then you could address these concerns and motivate CoralAgent. Also, it feels unnecessary to present C-STAR as a contribution independent of CoralAgent, as C-STAR is just a collection of information and results about the coral images presented at evaluation.

Response:

We thank the reviewer for these constructive suggestions and will revise accordingly. We will split Figure 2 into two figures with separate explanations and simplify Figure 3 to the core pipeline (point annotations → ecological targets → expert-authored questions). We will also use the three-site coral cover task as a running example in Section 3.

To clarify the sentence the reviewer quoted: the shared state is a structured record that all agents read from and write to, rather than passing results to each other directly. In the three-site coral cover task (Figure 1), the Vision agent writes the live-coral segmentation mask of each site together with the image and model that produced it. The Statistics agent reads these masks and writes the per-site cover estimates and point-count records (intermediate measurements). The Validation agent then compares the segmentation-based estimate with repeated point-count estimates and records whether each site passes (validation outcomes). If a site fails, recovery is triggered, for example by switching models or sampling additional points, and only revalidated estimates are used to compute the mean cover and identify the site with the lowest cover. We will rewrite this part of Section 3 accordingly.

In addition, we will also introduce Coral-Bench first to show the shortcomings of direct prompting and unconstrained tool use, which then motivate CoralAgent, and we will merge C-STAR into the description of CoralAgent rather than presenting it as a separate contribution. We will make all of these improvements in the revised version.

W5: Use of space in the main text

There are lots of moving parts in CoralAgent. There is an ablation study on which agents and contexts are necessary for performance, and it'd be a better use of space to discuss that in the main text rather than (1) describing the methodology in such length and (2) repeating a lot of numbers of Figure 4 in Section 5.2.

Response:

We thank the reviewer and agree. We will condense the methodology section and remove the numbers in Section 5.2 that repeat Figure 4. We will use the freed space to move the full ablation results (Tables 9 and 10) and their analysis into the main text. We will make these updates in the revised version.

AuthorsOpenReview ↗

CoralAgent: From Visual Recognition to Constrained Agentic Ecological Assessment for Coral Reef Conservation

Q1: Comparison with CoralVQA

Could you present some examples of how CoralBench has more interesting questions than CoralVQA, and why you think models do not succeed at those questions?

Response:

We thank the reviewer. CoralVQA mainly focuses on coral image understanding, asking single-step perception questions about one image, such as coral attributes, morphology or health status. Coral-Bench builds on such image understanding to evaluate real coral survey analysis tasks, which require multi-step quantitative analysis, for example:

“Which sites have live coral cover above the four-site average?” (Fig. 5)

“Between which adjacent survey years did live coral cover decrease the most?” (Fig. 5)

“What is the Gini–Simpson diversity index of hard coral genera in this coral survey area?” (Fig. 9)

Current VLM models show limited performance on these questions for three reasons.. First, the questions require measuring rather than describing, such as estimating the percentage of an image covered by coral or counting the coral genera present. Current VLMs are weak at such quantitative estimation; for example, they score near chance (9.6–25.2%) on coral cover estimation. Second, the questions require multiple steps, so errors accumulate: computing a diversity index involves identifying each coral, counting each genus and applying a formula, and a single misidentification carries through to the final answer. Third, the answers depend on precise ecological definitions; for example, bleached but living coral must still be counted as live coral, which models answering directly from the image often fail to apply consistently.

Q2: Human baseline

The questions in appendix (figure 5..9) look quite hard, even for coral ecologists. Could you provide a human baseline on this benchmark?

Response:

We thank the reviewer for this suggestion. We invited three marine scientists from our collaborating team, all holding PhDs in oceanography and with long-term experience in coral reef conservation, who were not involved in benchmark annotation. Each independently answered the family-level and genus-level recognition tasks:

Table R1: Accuracy (%) of human experts and CoralAgent on family-level and genus-level coral recognition.

TaskExpert 1Expert 2Expert 3Expert avg.CoralAgent
Family-level coral recognition91.8490.8293.8892.1880.61
Genus-level coral recognition84.2381.9886.4984.2361.71

The experts achieve high and consistent accuracy, which confirms that the questions are well-posed and answerable by domain experts. The remaining gap between CoralAgent and the experts (11.6 and 22.5 points, respectively) indicates substantial headroom, especially for genus-level recognition. Due to time constraints, the current human baseline covers only the coral recognition tasks; we will provide the human baseline for all tasks in the revised version of the paper.

AuthorsOpenReview ↗

CoralAgent: From Visual Recognition to Constrained Agentic Ecological Assessment for Coral Reef Conservation

Q3: Comparison with a general-purpose agent (Codex + GPT-5.4)

Since the proposal is a CoralAgent, how does an agent (codex + gpt-5.4) perform on this tasks? Appendix A.3.2. does this ablation but on a small subset of tasks from CoralBench.

Response:

We thank the reviewer for this suggestion. Since Codex no longer supports GPT-5.4 (it was retired from Codex on August 31, 2026), we were unable to evaluate the exact Codex + GPT-5.4 configuration. Instead, our GPT-Agent baseline is a Codex-style general-purpose agent built on GPT-5.4: it autonomously plans and invokes tools without domain-specific task constraints. It has access to the same tools as CoralAgent (coral-specific segmentation and recognition models, VLM-based visual analysis, and numerical calculation). Using the same GPT-5.4 backbone as CoralAgent enables a fair comparison between the two agents. Following the reviewer’s suggestion, we extended the full ablation from one task to 11 tasks, covering cover estimation, recognition, community composition and diversity:

Table R3: Extended ablation on 11 CoralBench tasks (accuracy, %).

TaskGPT (no tools)GPT-Agent + toolsGPT-Agent + tools + expert rulesCoralAgent w/o recoveryCoralAgent w/o validationCoralAgent
Coral cover estimation20.0054.0754.0771.8569.6372.59
Coral composition analysis27.1625.9320.9928.4029.6343.21
Bleached coral cover estimation14.0614.0637.5053.1348.4456.25
Coral richness assessment19.7525.9325.9351.8525.9355.56
Genus-level coral recognition42.3442.3447.7558.5658.5661.71
Family-level coral recognition68.3768.3782.6572.4578.5780.61
Dominant genus recognition56.7920.9920.9949.3858.0265.43
Shannon diversity index23.2919.1836.9943.8434.2552.05
Simpson diversity index19.1827.4027.4038.3649.3261.64
Species-level coral recognition52.0052.0053.3358.0058.0060.00
Pielou’s evenness index49.3234.2521.9250.6839.7361.64

These results support three findings:

  1. Tool access alone does not help.GPT-Agent fails to improve over tool-free GPT on 8 of 11 tasks, and drops sharply on dominant genus recognition (56.79 → 20.99). Successful tool calls can return unreliable outputs that propagate into downstream computations.

  2. Expert rules as context are not enough. Adding expert rules to GPT-Agent helps on some tasks but hurts on others, such as Pielou’s evenness and composition analysis. On family-level recognition, it slightly exceeds CoralAgent (82.65 vs. 80.61), likely because the rules directly support taxonomic identification. Overall, however, it remains well below CoralAgent, which outperforms it on 10 of 11 tasks with the same tools and expert rules.

  3. C-STAR’s validation and recovery are both necessary. Both mechanisms are specified by C-STAR, through its validation criteria (V) and recovery policy (R). Removing either one lowers accuracy on all 11 tasks.

We will include the full ablation in the main text and extend it to all 23 tasks in the revised version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.