- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes a shellfish benchmark for visual recognition of marine molluscs. This benchmark is large and comprehensive, including both visual images and a textual corpus. Based on that, the authors benchmark this dataset with various tasks, including closed-set recognition and image-to-image retrieval. Besides, they also examine the taxonomic error structure and controlled condition sensitivity. To sum up, this benchmark is practical and will be valuable for potential multimodal research in field in the future.
Critical strengths
- This dataset may be the largest publicly available curated dataset of molluscs to date. The authors have integrated data from multiple existing databases and other sources, carefully organized and cleaned the collection, and provided corresponding textual descriptions through captioning. These paired image–text data would facilitate future multimodal research on molluscs, making the dataset highly valuable for practical use.
- The authors conducted extensive benchmarking on this dataset, evaluating nearly 80 backbones to assess its difficulty. These experiments characterize the range of performance achieved by existing models and provide the first systematic assessment of how current state-of-the-art models perform in this specialized domain. This extensive evaluation is also one of the paper’s concrete contributions.
Critical weaknesses
None.
Questions for authors
- What does observation group mean? I could not find a definition in the paper. If I have overlooked it, please point me to the relevant explanation and clarify the term’s meaning and significance.
- I am interested in the details of the dataset deduplication procedure. Although the authors provide a general overview in the appendix, the specific steps are not described in sufficient detail.
- I am curious about the dataset’s level of fine-grained detail and the similarity between classes. The authors mention that some classes may be highly similar but do not provide supporting evidence. If possible, could the authors provide a confusion matrix to illustrate this?
4 comments
Response to Reviewer j5ep (2/3)
Q2. I am interested in the details of the dataset deduplication procedure.
Although the authors provide a general overview in the appendix, the specific steps are not described in sufficient detail.
The procedure combines exact-image checks, transformed-image candidate retrieval, local verification, and conservative exclusion before partitioning. We give the operational steps below, supplementing the audit totals in Appendix A.2 and Table 4.
-
Detect exact copies. We detect duplicate images by comparing file-level hashes and decoded-pixel hashes. The latter include image shape and RGB pixels after EXIF-orientation correction, detecting identical pixels stored in different files.
-
Retrieve possible transformed copies. We compute 64-bit perceptual hashes (pHash) and difference hashes (dHash). Views include the full image and an 82% central crop, each rotated by 0°, 90°, 180°, and 270°. Candidate retrieval uses four 16-bit pHash segments, requiring at least two shared-segment bucket occurrences; buckets exceeding 500 images are skipped. Global similarity uses HSV histograms and resized grayscale images. Hash distances are minima across the image views. A permissive filter retains exact matches or pairs satisfying any of four conditions:
- pHash distance ≤14;
- pHash ≤20 with global similarity ≥0.82;
- dHash ≤18 with global similarity ≥0.86;
- global similarity ≥0.965.
These rules select candidates for verification.
-
Verify local correspondence. We extract up to 3,000 ORB features per image and apply a 0.75 nearest-neighbor ratio test. RANSAC estimates a homography with a 4-pixel reprojection threshold. Verification images have a maximum long edge of 1,200 pixels. We measure inlier count, inlier ratio, spatial coverage, and photometric normalized cross-correlation (NCC) after alignment. Exact file or decoded-pixel matches are accepted directly. Other pairs must satisfy every threshold in the applicable column below.
| Final duplicate criterion | Same scientific name | Different scientific names |
|---|---|---|
| Minimum pHash Hamming distance | ≤6 | ≤2 |
| Minimum dHash Hamming distance | ≤8 | ≤2 |
| RANSAC inliers | ≥45 | ≥45 |
| Inlier ratio | ≥0.55 | ≥0.80 |
| Spatial coverage | ≥0.20 | ≥0.30 |
| Aligned photometric NCC | ≥0.90 | ≥0.85 |
-
Separate decisions and select representatives. Pairs are classified as strict duplicates, unresolved candidates for review, or rejected duplicate hypotheses. Strict duplicate pairs form clusters, with one representative retained per cluster. Representative selection considers license eligibility, rights verification, original-source status, resolution, sharpness, watermarks, and provenance completeness. Unresolved candidate images are conservatively excluded from release selection. Pairs whose duplicate hypothesis is rejected retain both images.
-
Audit the release and partition by group. After deduplication, selection applies the quality, licensing, class-size, and group constraints. We verify that excluded records are absent from both the selected and released collections. We then check the released record correspondence and group-disjoint partitions.
The audit covers 57,171 candidate images. Candidate retrieval produced 2,106,464 pairs, of which 194,024 underwent local verification. The final exclusions and release checks are:
| Audit item | Result |
|---|---|
| Strict duplicate clusters | 216 clusters containing 442 images |
| Nonrepresentative cluster members removed | 226 images |
| Additional unresolved candidate paths excluded | 871 paths |
| Unique exclusions absent from selection and release | 1,097 / 1,097 |
| Selected-to-released record correspondence | 40,130 / 40,130 |
The 871 additional exclusions come from 878 paths in 754 unresolved pairs, after removing seven paths already counted among the strict removals. The record-correspondence check uses stored checksums and source-identity fields. These checks connect the deduplication decisions to the released dataset and confirm zero observation-group overlap across partitions.
Response to Reviewer j5ep (3/3)
Q3. I am curious about the dataset’s level of fine-grained detail and the similarity between classes.
The authors mention that some classes may be highly similar but do not provide supporting evidence. If possible, could the authors provide a confusion matrix to illustrate this?
ShellfishNet tests species identification from ordinary photographs. The confusion analysis identifies retained species that remain difficult to distinguish across models and seeds. We analyzed full 100-class test confusion matrices for MaxViT-Tiny and DINOv2 ViT-B/14. We normalized each row, then averaged the matrices across seeds 41, 42, and 43 on the same 3,958 test images. Rows denote true species and columns denote predictions.
To examine specific confusions, we pooled the six validation confusion matrices from these two models and ranked species pairs by symmetric row-normalized confusion. We selected the ten highest-ranked pairs. We then measured their confusion on the held-out test set. All counts come from full 100-class predictions. For seed , the symmetric confusion rate is
where counts true class- images predicted as , and is the class- test count. Larger values indicate greater confusion between the two species. The table reports three-seed mean in percent:
| Retained species pair | Test images ( / ) | MaxViT-Tiny (%) | DINOv2 (%) |
|---|---|---|---|
| Busycon carica / Sinistrofulgur sinistrum | 47 / 42 | 14.94 | 16.22 |
| Spisula solidissima / Tivela stultorum | 47 / 45 | 13.30 | 9.24 |
| Crassostrea virginica / Ostrea edulis | 49 / 49 | 7.48 | 7.82 |
For Busycon carica and Sinistrofulgur sinistrum, errors occur in both directions across all 23 original models and three seeds (69 runs). This repeated confusion provides direct evidence of species-recognition difficulty within the selected inventory. Appendix G.1 reports all ten validation-selected pairs in Table 25 (p. 52) and their individual-seed results in Figure 21 (p. 53). Figure 22 (p. 53) adds directional confusion matrices for the seven species in pairs 1–4. Rows give true species and columns give predictions; the “Other 93” column retains errors to the remaining classes. Each row sums to 100% before rounding. These named pairs make the recurring identification difficulties explicit.
Dear Reviewer j5ep,
We hope you are well. Thank you again for your careful review and helpful questions.
Our responses explain the observation groups and deduplication procedure, and the revised appendix provides additional evidence of interspecies confusion. When convenient, we would welcome your feedback on whether these explanations address your questions. If anything remains unclear, please let us know; we would be happy to clarify.
Thank you for your time and consideration.
Best regards,
The Authors
Response to Reviewer j5ep (1/3)
We thank the reviewer for the detailed feedback and for recognizing the value of the dataset and benchmark. We clarify observation groups, detail the deduplication procedure, and present species-level confusion evidence below. Additional analyses are provided in Appendix G.1–G.6 of the revised manuscript.
Q1. What does observation group mean?
An acquisition or observation group links images from the same capture event or source record. For example, two photographs attached to one observation belong to the same group. Section 3.1 (pp. 4–5) defines this term and explains the partitioning rule; Appendix A.2 (pp. 23–24) describes the released grouping metadata.
We construct the group identifier from the first available field in this order: acquisition-session identifier, observation identifier, specimen identifier, and source-platform plus source-record identifier. The released metadata stores this assignment as
group_id.Each group is assigned wholly to training, validation, or test. This keeps related views together and reduces leakage from shared subjects or capture conditions. Deduplication removes image copies; grouping keeps the remaining related photographs in one partition. ShellfishNet contains 40,130 images in 30,879 groups, with at most two images per group. No group crosses partitions. Figure 2(d) shows the group-size distribution, and Table 4 (p. 24) reports the partition audit.