- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces SHIFT-Truck, a high-fidelity CFD dataset designed to address the lack of training data for aerodynamic surrogates for pickup-truck. It’s motivated by the practical needs in vehicle design, where pickup trucks represent a major vehicle category, however, was not included in existing automobile datasets for aerodynamic surrogates. SHIFT-Truck contains 800 simulation cases covering 17-parameter design space, and it describes the time-averaged flow. The authors benchmark 4 recent neural surrogate architectures using geometry-grouped splits and conduct out-of-distribution evaluations across freestream speeds, discretization, and extra vehicle types. Authors also analyze the observed model failures with discussion of limitations and planned extensions of the dataset.
Critical strengths
- The dataset is well motivated and directly connected to practical vehicle design, as authors claimed, it bridges an existing gap. Its detailed flow fields and controlled changes in geometry and parts provide a useful resource for learning aerodynamic surrogates.
- The paper is quite easy to follow, the dataset design involves large number parameter space, but the illustration figures handle this elegantly, which helps to see how the cases are constructed and what variations the benchmark covers. In addition, released samples give a straightforward view of the dataset.
Critical weaknesses
- Out-of-distribution evaluations across freestream speeds and surface discretizations use a single training-set truck, making conclusions drawn from there less supportive.
- As authors acknowledged in the limitations, the dataset contains only time-averaged fields, restricting its applicability to tasks involving unsteady loads and vortex shedding.
Questions for authors
- Could authors report more training configurations and details in the appendix? Like GPU memory consumption, batch size, and model training/validation curves. These would clarify the computational requirements to use the dataset and whether 500 epochs are sufficient for each model.
- Can make some figures in the main text more compact, leaving and taking usage of room for important material currently in the appendix? Like Table 11 which could help understand where SHIFT-Truck differs from existing datasets and what it contributes. The appendix is rich but main text should be able to fit more content than current.
- For OOD evaluations, for the study across freestream speeds and discretizations, could authors include a few more truck geometries if time allows?
3 comments
Thanks for the detailed response and the extended experiments. My questions are addressed and I think it’s a good dataset and benchmark paper. Please consider upload rebuttal revision PDF if time allows. I will increase my score.
My final recommendation stays as Clear Acceptance. As a Dataset and Benchmark paper, this paper includes large number of evaluated methods, and the dataset design is well motivated with practical usage. It will be valuable to the community after being publicly accessible. Authors provided extra benchmarks of recent models during the limited rebuttal period and further clarified novelty and issues other reviewers asked for. It will be good to see other reviewers' participation during this discussion period.
Response to Reviewer 3QLD
W1 and Q3. OOD studies on more trucks. We have extended both studies.
Discretization. Mean |CD error| over ten held-out test trucks, in counts (1 count = 0.001), by fraction of surface faces retained:
Sensitivity to the input discretization depends strongly on the architecture, as in the single-truck study.
Speed. The speed study now covers four configurations, with the bed open and closed and the chin spoiler on and off. Mean |CD error| in counts over the four:
The behaviour reported in the paper holds on each configuration. Both studies will be in Fig. 6 of the camera-ready version (https://github.com/shift-truck-submission/review-3QLD/blob/main/figures/fig6.png).
W2. Time-averaged fields. We agree that unsteady loads and vortex shedding are important targets. Every case was run as a transient simulation and the time-resolved fields exist, but their size keeps them out of the release: the time-averaged fields of the 982 cases already take about 11.6 TB, and time-resolved volume fields would reach hundreds of terabytes. We are preparing a smaller transient subset as part of the planned extension (§6) and will add further transient samples as the community requests them.
Q1. Training details. Every model trains with one case per GPU on eight 80 GB H100 GPUs (an effective batch of eight cases), for 500 epochs in FP32:
The loss curves flatten well before the end of training: validation loss changes by only 0.6–1.4% between epochs 300–399 and 400–499, so 500 epochs are sufficient. In the camera-ready version, the table will go into Table 8 and the training and validation curves into Appendix E (Fig. R2, https://github.com/shift-truck-submission/review-3QLD/blob/main/figures/figR2.png).
Q2. Layout. In the camera-ready version, Table 11 will move into §1 next to the contributions, together with the shape-embedding comparison with DrivAerML and SHIFT-SUV, so the introduction shows where SHIFT-Truck differs from existing datasets. To make room, we will compact two main-text figures and use the additional page.