acceptodds
Under review as a conference paper at ICLR 2027

SparseWake: A Benchmark for Multi-Source State Estimation from Local Flow Measurements

Abstract

Local flow sensing can help underwater robots locate a target of interest, but the measured signal is shaped by nearby sources and may not be dominated by the target. A controlled testbed with source-level records would allow us to diagnose where localization fails and to isolate the effect of source competition, guiding the design of local perception for future underwater agents. Here, we introduce SparseWake, a benchmark for state estimation from sparse, body-oriented velocity histories generated by a physics-informed dynamic model of fish wakes. It defines three task families of increasing competition. A single-neighbor task (canonical) establishes a reference, controlled mixtures add interference around a fixed nearest target, and symmetric source sampling (common-prior) selects the nearest source only after the scene is generated. Alongside the mixed observations, SparseWake retains each source's flow contribution, so that predictions can be diagnosed against source-level ground truth and individual contributions can be edited without changing the target. To demonstrate its use, we evaluate four architectures and find a shared failure mode, in which predictions far from every source dominate squared localization error. We further characterize the roles of temporal and measurement structure, and use paired source interventions to show that fitted estimators respond to relative source contribution beyond overall signal scale. We package the observations, source records, splits, baselines, and evaluation tools as a controlled testbed for developing local perception in underwater agents.

Primary area:
datasets and benchmarks
Keywords:
hydrodynamic sensing, multi-source localization, inverse problems, time-series learning, synthetic benchmarks, underwater perception

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZL1XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a synthetic benchmark, SparseWake, for studying whether an underwater robot can estimate the state of nearby fish using sparse local flow measurements. The main challenge is the multi-source case, where several fish contribute simultaneously to the measured flow. The benchmark is carefully constructed and provides useful source-level information that enables controlled analysis of interference and localization failures. The authors evaluate the effects of temporal history, sensor configuration and measurement noise using several baseline architectures. The results show that most errors arise from far-away sources rather than localizing to the wrong source.

Critical strengths

The common prior task is very well motivated and carefully designed. Instead of sampling a privileged target, all sources come from the same distribution, and the selection happens afterwards. The source decomposition is also useful for diagnosis and is able to generate experiments that are difficult to perform with real data.

Critical weaknesses

One major weakness is that the authors do not attempt to transfer to real data. All train/test observations are produced by the same physics-informed wake generator. Evaluation of an out-of-model flow would substantially strengthen the contribution of the paper, but the authors explicitly state that such experiments are not available.

The main empirical result that most of the localization errors come from predictions far from every source, rather than the model selecting the wrong fish, is acknowledged but left unresolved. Utilizing a multi-hypothesis baseline will allow to draw conclusions about whether a model failure is due to intrinsic ambiguity or the claimed conditional averaging.

Questions for authors

Look above

6 comments
AuthorsOpenReview ↗

Response to Review Comments 1/2

Thank you for your constructive feedback. These comments led us to make the physical evidence more explicit and to test the multi-source prediction approaches. We respond to the experimental-data concern first, then discuss what the new failure analysis shows. Page references are to the revised manuscript.

1. Connect the benchmark's physical foundation and scale to the challenges of obtaining experimental learning data.

To address this reasonable concern, we now add a clearer account of the published DNS evidence and the challenges of collecting experimental benchmark data.

The flow generator was built based on the hydrodynamic model of Zhou et al., Physics of Fluids (2025). That study used high-fidelity three-dimensional direct numerical simulations (DNS) to parameterize wake geometry and circulation decay. It also validated model-reconstructed vortex organization and time-averaged induced flow with DNS. We now add this inherited physical evidence in Section 3.1 (p. 3, bottom) and Supplement S1.1 (p. 13).

We also revised the Discussion to explain the practical gap between measuring a school and assembling a learning benchmark. Pan and Lauder (2024) demonstrate multi-fish particle image velocimetry (PIV), but report blocked illumination and camera occlusion within schools. These effects leave parts of the flow unmeasured and complicate synchronized flow and state recording. Building a typical sized benchmark also requires coverage across configurations with consistent labels and evaluation splits, which is extremely challenging with experimental swimmers and current techniques. We indeed plan to incorporate experimental measurements and independent DNS data in future work, as advances in PIV and lower simulation costs make systematically collected, well-labeled datasets more accessible.

Our common-prior task supplies 16,000 training scenes and separate validation and test sets (Table 1, p. 4). Because we keep each source's contribution, we can change interference while holding the geometry and target fixed. We demonstrate these controlled tests in Section 5.3 (pp. 9–10). The revised Discussion explains this contribution and identifies physical-sensor and free-swimming experiments as the next tests of transfer (p. 10). The published DNS evidence supports the inherited flow construction.

(Due to the length limit, continue on the next comment thread.)

AuthorsOpenReview ↗

Response to Review Comments 2/2

2. Can multi-hypothesis prediction explain the far-from-all failures?

Following your suggestion, we added a multi-hypothesis comparison that allows several candidate locations for the nearest target. If we understood your suggestion correctly, is testing whether forcing an estimator to return one position contributes to the far-from-all errors. Besides, we also expanded the spatial diagnostics and evaluated joint source reconstruction. The three analyses address different parts of this question.

(a) We tested multiple candidate target locations.

We implemented the multi-hypothesis baseline with a five-component Gaussian-mixture head. Each component provides a candidate location with a predicted weight and spread. We compared it with matched point-regression and single-Gaussian heads on each of the same three frozen MLP encoders, giving nine newly trained heads. All heads use the same learned features and nearest-target position supervision, and we selected them using validation data and evaluated them on both common-prior test sets. This comparison helps isolate the effect of the output head.

The table reports the additional test set. Fish counts include the observer, and where L is the reference source-body length. Position metrics compare the point head with the mixture mean, and negative log-likelihood (NLL) compares the Gaussian with the mixture density. Lower RMSE and NLL, and higher hit rates, are better.

FishRMSE / L: point → mixture meanHit within 0.25 L: point → mixture meanNLL: Gaussian → mixture
30.4680 → 0.459763.57% → 67.96%−0.3390 → −0.9145
50.8032 → 0.811537.15% → 39.28%1.5019 → 0.6822

The mixture improves probability scores and hit rates for both fish counts. Three-fish RMSE decreases, but five-fish RMSE increases. We also select the highest-weight component's mean as a single candidate, avoiding an average across components. This raises hit rates to 69.34% and 40.15%, with RMSEs of 0.4872 and 0.8716 L. This summary of the experiment has been updated in Section 5.1 (p. 8), the protocol and results in Supplement S5.6 (pp. 28–31, Tables S20–S21).

(b) We checked whether failed predictions lie near other fish or source-position averages.

For seven reference estimators on the additional test set, only 0.21–4.70% of failed predictions lie within 0.25 L of another actual source. Among the four neural models, the category near an equal-weight source-subset mean, while remaining far from every actual source, accounts for 3.97–4.98% of three-fish failures and 14.42–15.12% of five-fish failures. These percentages use failed predictions as the denominator.

Now these geometric averages are distinguished from conditional averaging over possible targets in different hidden scenes compatible with an observation. The spatial categories describe where errors occur. The explain has been added to Supplement S5.4 (p. 27, with Table S18 on p. 28).

(c) We tested whether reconstructing all sources improves nearest-target localization.

We trained joint source-set MLP and TCN models with known source count and all-source position supervision. On the original test set, two-source prediction improves hit rates but increases nearest-target RMSE. With four sources, both metrics worsen. This tests recovery of one complete source configuration, a different output task from proposing alternative locations for the nearest target. We report it in Supplement S3.5 (pp. 20–21, Table S10).

The multi-hypothesis comparison shows that changing the output formulation improves some prediction scores. However, far-from-all predictions still contribute 88.92% and 88.86% of the mixture mean's squared target-position error for three and five fish. Distinguishing ambiguity in the measured flow from limitations of the frozen representation or fitted head remains an open question. We discuss this interpretation in Supplement S5.6 (p. 31).

Thank you for suggesting this comparison. If we have misunderstood your concern or the intended multi-hypothesis test, we would appreciate your clarification so that we can address it during the rebuttal period and further improve the manuscript.

AuthorsOpenReview ↗

Update on Real-World Data Alignment

While you are reviewing our new updates according to your initial review, we are happy to share the good alignment between our benchmark and experimentally measured flow magnitudes. Thus, we believe these experiments significantly helped us improve our work and strengthen the experimental basis of our paper.

We examined both datasets. The Wolf and van Netten (2019) data contain sensor-deflection signals, so a verified calibration is needed for a direct velocity comparison. Thus, we compared the fish PIV dataset associated with Tack et al. (2024) with our flow generator with three recordings from three individual fish swimming at a nominal speed of 1 body length per second.

We first compared the mean flow fields and sampled corresponding locations on both sides of each fish. Across the three recordings, the mean absolute differences in streamwise and lateral flow velocity were 4.99% and 3.02% of the nominal swimming speed, respectively. We also compared temporal velocity fluctuations. The results showed that their spatially averaged RMS amplitudes differed from the PIV measurements by 1.81–6.15%, relative to the measured amplitudes. These results indicate good agreement between the synthetic and experimentally measured flow magnitudes at the sampled flank locations.

We have added these results in Section 3.1 in the main text and a new supplementary section, “Comparison with experimental flow measurements” (Supplement S8), including the details of the comparison procedure, flow-field plots, and quantitative tables.

Reviewer_kYkrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a benchmark dataset for estimating fish poses based on locally observed flow measurements. The dataset is based entirely on data simulated using mechanistic models. The benchmark contains three tasks that progressively become more difficult. The benchmark includes seven baselines, four of which are based on neural networks.

Critical strengths

I find the paper to be very well written. The problem is clearly motivated and the benchmark thoroughly described. The set of baselines is well chosen and includes relatively simple baselines such as the count-informed constant, which are useful for sanity-checking.

Critical weaknesses

The paper is currently somewhat unclear about how distinct the train/test sets are (poses are different, but how different?). Is generalization beyond the train set even important, or would interpolatation between training examples be sufficient? I think these questions are particularly critical especially as more capable and overparameterized models might be applied to this dataset in the future.

Questions for authors
  • Why is the task formulated as locating the nearest source and not as localizing all sources?
  • Can you clarify how you generate multiple samples/predictions (e.g., in Figure 3) while keeping the measurements and model fixed (I assume)?
  • Can you clarify what you mean by "original protocol", e.g., in "The original protocol also held test poses apart, but allowed different samples from the same pose to appear in training and validation."
  • Why not include confidence intervals in Table 2?
  • Can you clarify what is meant by "test population" and how the two test populations differ?
3 comments
AuthorsOpenReview ↗

Response to Review Comments 1/2

Thank you for your thoughtful questions and for recognizing the value of the SparseWake. To address your concerns, we added a geometric comparison of training and test scenes, tested all-source prediction, and clarified the evaluation protocol. Page references are to the revised manuscript.

1. How training and test configurations differ, and how interpolation fits the benchmark's goal?

We have updated our manuscript to further clarify it. We added training–test geometry distances and clarified that interpolation is a valid solution to the core task.

In the common-prior task, we sample all sources from the same proposal distribution and select the nearest afterward. Training and test scenes come from the same sampler and flow model. Scenes are kept complete and include their phase histories in separate splits. This allows larger models to be evaluated on the same held-out configurations. We explain the target rule in Section 3.2 (p. 4) and the evaluation scope in Section 3.3 (p. 5, “Common-prior evaluation”).

We also compared each test scene with the training set using both target position and the complete arrangement of source positions. The complete-scene distance accounts for permutations of source identities. Median distances are:

Nearest-training distance, in LThree fishFive fish
Target position alone0.01310.0121
Complete source-position configuration0.17660.5008

Fish counts include the observer. A nearby training target can therefore have different surrounding sources and a different mixed flow. The complete-configuration distances are comparable to those between distinct training scenes. We provide definitions and results for both test sets in Supplement S2.4 and Table S4 (p. 17).

2. Why select the nearest source instead of predicting all sources?

This is a wonderful question! We chose the nearest target as guided by biological observations. Herbert-Read et al. (2011) found a dominant nearest-neighbor response in mosquitofish. Mekdara et al. (2021) linked lateral-line function to tail-beat synchronization in schooling fish. These observations motivate the nearest neighbor as the local tracking priority. In our task, every source contributes to the observation, so recovering the nearest source still requires separating geometric relevance from signal strength. Now, we include this in the Introduction (p. 2, upper part), and the biological context in Section 2 (p. 2).

Following your question about all-source localization, we trained MLP and TCN for two and four external sources to predict all source positions using three seeds. These models receive all-source position supervision and know the source count. We evaluate full-set recovery and select the predicted source closest to the observer to measure nearest-target accuracy. With two external sources on the original test set, the MLP hit rate within 0.25L increases from 62.65% to 64.17%, but nearest-target RMSE increases from 0.4915 to 0.5693L. TCN shows the same tradeoff. With four sources, both metrics worsen for both architectures. We summarize the comparison in Section 4.1 (p. 6, below Table 2). Supplement S3.5 reports both evaluation targets, with Table S10 on p. 21. These results suggest that the models predicting all sources don't consistently improve, if not hurt, the performance of locating sources.

(Due to the length limit, continue on the next comment thread.)

AuthorsOpenReview ↗

Response to Review Comments 2/2

3. How are multiple predictions generated for a fixed scene in Figure 3?

We now clarify that the predictions vary across input histories and trained checkpoints. Each scene supplies 32 cyclic, 24-frame histories, one ending at each stored phase. Three independently trained MLP checkpoints therefore give 32 × 3 = 96 predictions per scene. For a given history and checkpoint, the prediction is deterministic. We describe the history construction in Section 3.1 (p. 4, immediately before Section 3.2). Figure 3's caption identifies the phases and seeds (p. 8), and Supplement S5.4 specifies all three seeds and 32 phases (p. 27).

4. What does “original protocol” mean?

We now name and describe the earlier sample-level split and the pose-disjoint split separately. In the earlier protocol, all endpoints from 1,000 of 5,000 poses were held out for testing. Endpoints from the other 4,000 poses were divided between training and validation, so a non-test pose could appear in both. The main architecture comparison uses disjoint groups of 3,400 training, 600 validation, and 1,000 test poses.

We have clarified this in Section 3.3 (p. 5, “For the canonical architecture comparison”), with details in Supplement S2.1–S2.3 (pp. 14–16).

5. How do the two test populations differ?

In our updated manuscript, we identify the original test split and the additional test set explicitly, including which one supplies Table 2. Each contains 4,000 scenes: 2,000 three-fish and 2,000 five-fish scenes. Both use the same sampler and flow model. The additional set has separately sampled geometries and initial phases, shares no geometry with the data bank, and supplies the common-prior results in Table 2.

These definitions appear in Section 3.3 (p. 5), the Table 2 caption (p. 6), and Supplement S2.1 (pp. 14–15).

6. Why were confidence intervals absent from Table 2?

Thanks for pointing it out. We now add 95% confidence intervals across 2,000 bootstrap draws that resample complete poses or scenes in Table 2 (p. 6).

Thank you again for pointing out where the evaluation needed clearer explanation!

AuthorsOpenReview ↗

Gentle Reminder and Revision Summary

We thank the reviewer again for the positive assessment and thoughtful questions. As the discussion period is ending, we briefly summarize our previous responses below:

We clarified:

  1. The training and test configurations, how the two test populations differ, and how interpolation fits the benchmark’s goal.
  2. The rationale for targeting the nearest source rather than all sources is now explained in the manuscript. In addition, further details on how multiple predictions are generated and what “original protocol” refers to are now provided.

Beyond these clarifications, the revised version includes other important improvements, for example:

  1. Our benchmark, SparseWake, shows good agreement with experimental PIV measurements.
  2. Extended and evaluated the dataset that includes different swimming speeds, relative sizes, and motion within a history

We hope we have addressed all your questions, and we would be grateful if you could kindly consider increasing the score to reflect the improvement.

Reviewer_QkkgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a benchmark for state estimation from sparse, body-oriented velocity histories generated by a physics-informed dynamic model of fish wakes. It includes a data generator and observation interface, and three types of tasks. The most prominent data problems are sparse measurements and multi-source measurements, which contaminate the data.

Critical strengths

The authors release datasets, source-level records, generation code, reference checkpoints, and evaluation tools.

Critical weaknesses

The topic could be more interesting to a more focused audience.

The paper does no real-world validation. The benchmark is purely synthetic. There is no evaluation on real lateral-line and flow-sensor data, no transfer study to tank experiments or in-water robots, and no quantification of the sim-to-real gap. The reduced-order wake model is taken as ground truth with no sensitivity analysis to model error.

Limited kinematic and morphological diversity. Body length and nominal speed are fixed to L=1. What would happen if different sizes of fish were present in the simulation?

Within a history, the relative pose and heading seem to be constant.

What is the leakage effect of training set normalization on test set distribution, in view of cyclic wrapping of the data?

Baselines are not challenging. The evaluation focuses on standard regression architectures (MLP, TCN, GRU, and Transformer), alongside simple ridge-regression and retrieval baselines. It does not investigate methods for inverse problems, source separation, or multi-source state estimation. This is important because the benchmark's main challenge is separating overlapping physical signals. Without stronger, problem-specific baselines, it is difficult to determine whether the observed localization failures reflect fundamental limitations of the sensing problem or simply inadequate modeling approaches.

Faillure analysis is not informative. failures arise from insufficient observability, ambiguity between source configurations, or regression models averaging over multiple plausible positions?

Questions for authors

See weaknesses

6 comments
AuthorsOpenReview ↗

Response to Review Comments 1/2

Thank you for your constructive feedback. We respond below to your concerns about physical grounding, variation across conditions, data leakage, and the interpretation of failures. Page references are to the revised manuscript.

1. Explain the physical foundation and broader value of the benchmark more clearly.

We use the hydrodynamic construction of Zhou et al., Physics of Fluids (2025). This study parameterized wake geometry and vorticity circulation decay using high-fidelity three-dimensional DNS, then validated the model-reconstructed flow with DNS. We now describe this inherited evidence of our physical foundation in Section 3.1 (p. 3, bottom) and Supplement S1.1 (p. 13).

We now explain the realistic challenges in obtaining real-world flow measurements to build a learning benchmark in the updated manuscript. Pan and Lauder (2024) demonstrate multi-fish particle image velocimetry (PIV) and describe how bodies block illumination and camera views. Across many configurations, these gaps complicate simultaneous flow and state measurement. Our common-prior task supplies 16,000 training scenes with synchronized labels and separate evaluation sets (Table 1, p. 4). We also change a source's contribution while keeping the geometry and target fixed. This allows researchers to test how an estimator responds to interference in overlapping physical signals. We have clarified this broader use explicitly in the Introduction (pp. 1–2) and Discussion (p. 10).

2. Expand and evaluate variation in swimming speed, relative size, and motion within a history

This is a good point. To address it, we generated speed, relative-size, and motion evaluation subsets and tested them with the existing MLP checkpoints, which directly respond to the questions about fixed nominal scales and motion within a history. % They complement the existing sensing-layout, noise, source-count, and wake-expansion studies.

L = 1 defines the reference source-body length. To vary relative size, we use observer/source length ratios of 0.8, 1.0, and 1.2 and scale accordingly the coordinates of the observer's six probes. To vary source speed, we regenerate the wake and potential flow at U/U0 = 0.6, 0.8, 1.0, 1.2, and 1.4. Two speed protocols and an amplitude-only control distinguish changes in wake geometry from signal scaling (Supplement S6.4, p. 34). The earlier wake-expansion study is in S6.2 (p. 32).

We then evaluate matched held-out poses within each family using the same three pretrained single-neighbor MLP checkpoints and their training normalization. This measures direct transfer as each variable changes. Position RMSE is in reference source-body lengths L:

Added evaluationRepresentative RMSELocation
Source speed, fixed model ratios0.0625 at U/U0 = 1.0, 1.0873 at 0.6, 0.6668 at 1.4Table S27, p. 34
Observer/source length ratio 0.8, 1.0, 1.20.2321, 0.0625, 0.1953Table S26, p. 34
Prescribed relative motion0.2320–0.3943, versus 0.0623 for matched static historiesTable S28, p. 35

The relative pose and heading do stay fixed within a history as wake phase advances. For extending the motion histories, we choose the final state and integrate backward to obtain the preceding positions. The moving and static histories therefore end with the same observation and label. We included these in Section 4.3 (p. 7, “Parameterized evaluation extensions”), with protocols and confidence intervals in Supplement S6.4 (pp. 32–35).

(Due to the length limit, continue on the next comment thread.)

AuthorsOpenReview ↗

Response to Review Comments 2/2

3. Clarify how normalization, cyclic histories, and data splitting interact.

We have clarified how normalization and cyclic histories respect the split, and report a matched comparison of the two validation protocols in the updated manuscript.

To address your concerns about whether wrapping a history could allow information from test scenes to enter training, we compute normalization statistics from training data and freeze them for validation and testing. Wrapping uses phases from the same scene. Complete test scenes remain separate from training and validation, so wrapping does not cross that boundary. We now state these rules together in Section 3.3 (p. 4, bottom, continuing on p. 5).

The earlier canonical sample-level protocol allowed non-test poses to appear in both training and validation. The pose-disjoint protocol separates those groups. With test poses and endpoint budgets fixed, the comparison changes MLP RMSE from 0.0575 to 0.0628 L. The paired increase is 0.0053 L, with a 95% interval of [0.0022, 0.0089] L. Because training-pose coverage also changes, this measures the combined effect of coverage and validation grouping. We explain the protocols in Section 3.3 (p. 5) and the result in Section 4.1 (p. 6, “Canonical accuracy also depends”). Supplement S2.1–S2.3 gives the details (pp. 14–16).

4. Add task-specific estimators and use them to investigate the localization failures.

Thank you for the suggestion. We have added physics-based inversion, joint source-set prediction, and a multi-hypothesis output comparison to examine whether using the forward model, predicting every source, or allowing several candidate locations changes the observed failures.

Our inverse estimator fits source configurations through the forward model under a fixed search budget. On a separate confirmation set of 16 scenes per source count, its RMSE is 0.608/0.807 L for two/four external sources. The lowest neural reference errors are 0.392 L for MLP and 0.678 L for TCN, respectively. External-source counts exclude the observer. We report this finite-budget comparison in Supplement S3.4 and Table S9 (pp. 19–21).

We also trained joint source-set MLP and TCN models using all-source position supervision and known source count. With two sources, they improve threshold hit rates but increase nearest-target RMSE. With four sources, both metrics worsen for both architectures. Section 4.1 summarizes these results below Table 2 (p. 6), and Supplement S3.5 and Table S10 give the full comparison (pp. 20–21).

To test whether a single point output contributes to the failures, we trained matched point, Gaussian, and five-component mixture heads on the same frozen MLP representations. On the additional test set, the mixture improves likelihood. Its mean changes three-/five-fish RMSE from 0.4680/0.8032 L to 0.4597/0.8115 L, counting the observer. We summarize this experiment in Section 5.1 (p. 8) and provide the protocol and results in Supplement S5.6 (pp. 28–31, Tables S20–S21).

We also refined the spatial analysis to distinguish predictions near other fish from predictions near averages of source positions. This explains why averaging fish positions in one scene differs from averaging possible targets across scenes compatible with an observation (Supplement S5.4, pp. 27–28). Under the mixture mean, predictions more than 0.25 L from every source still contribute about 89% of squared target-position error. Thus, changing the output distribution improves some scores, but the main error pattern persists. Distinguishing weak information in the observed flow from information lost by the encoder remains an open question, as discussed in S5.6 (p. 31).

Thank you again for your valuable comments which help us improve our manuscript!

I appreciate the additional experiments and clarifications.

I remain concerned about the absence of empirical testing. While I acknowledge the difficulty of collecting large-scale synchronized flow and state measurements from fish schools, this does not preclude using available empirical data to assess the physical validity of the benchmark, and assessing the sim to real gap of the proposed data.

For example, Wolf and van Netten (2019, DOI: 10.34894/SDETIR) provide publicly available collected measurements from a two-dimensional artificial lateral-line sensor array. Experimental fish-wake velocity fields and kinematic data are also available, for example, from Tack et al. (2023, DOI: 10.5281/zenodo.8249841). These references come from a simple google search, so other data can exist to help you quantify how far you are from real measurements.

These datasets do not reproduce the complete multi-source task, but they offer opportunities to assess whether the synthetic flow measurements are physically representative.

Moreover, the new sensitivity analyses show substantial deterioration under relatively modest variations in swimming speed, body size, and relative motion. This raises additional questions about the generality of the benchmark's conclusions beyond its nominal simulation conditions.

I recognize the contribution of a controlled synthetic benchmark, particularly its source-level annotations and intervention capabilities. Nevertheless, for a datasets-and-benchmarks contribution motivated by real hydrodynamic perception, I consider some independent evidence of physical representativeness important. Checking against the same family of numerical models used to construct the benchmark does not completely resolve this concern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.