acceptodds
Under review as a conference paper at ICLR 2027

SWIM: COMPACT ENVIRONMENT REPRESENTATION FOR REINFORCEMENT LEARNING HUMAN SWIMMING

Abstract

Deep reinforcement learning (RL) has driven rapid progress in physically-based motion generation, yet synthesizing robust motion policies in dense fluid environments (e.g., swimming) remains unsolved. Unlike land motion, where environmental impact is sparse and can be coarsely modeled (e.g., gravity, normal reaction), swimming requires continuous, full-body coordination under pressure and flow forces across the entire body surface; fully-coupled rigid–fluid simulation is far too slow for the millions of interactions RL requires. We propose SWIM, an RL framework for physically-based human swimming learned from a single reference motion. Its core is a compact, low-dimensional environment representation: a dual-branch graph-convolutional VQ-VAE that tokenizes per-link body–water forces and torques, informative enough for control yet robust to the rapidly changing force exchanges that would otherwise destabilize training. We pair this with a GPU-based Lagrangian solver with rigid–fluid coupling that yields 10–15× faster training. Across hundreds of training runs and several hundred zero-shot conditions, SWIM generalizes to unseen goals and trajectories, fluids with varying density, external flows and perturbations, altered body morphologies, and all four competitive swimming styles. Against imitation-learning baselines (MimicKit-DeepMimic, MimicKit-AMP, ADD) and alternative force models (a PINN world model, MuJoCo’s inertial fluid model, and an underwater-robotics simulator), SWIM achieves better stability, goal satisfaction, and physical realism: a 63.9% success rate on held-out generalization versus 36.8% for the strongest baseline.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Motion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_miuxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SWIM is a reinforcement-learning framework for physically simulated human swimming. Its main component represents body–water interactions using whole-body forces and torques together with per-link features encoded by a dual-branch GCN VQ-VAE. A PPO policy adjusts stroke frequency and reference-pose offsets, while a GPU-based fluid–rigid simulator supports training. Experiments cover four swimming styles and generalization to new paths, flows, fluid properties, perturbations, and body geometries.

Overall, I am slightly positive. SWIM presents a useful system for a challenging swimming-control problem, and the broad generalization experiments are a meaningful strength. My main reservation is that the necessity and advantage of the proposed tokenizer remain unclear relative to simpler force representations. Nevertheless, the system-level contribution and demonstrated capabilities are sufficient to support weak acceptance.

Critical strengths
  1. The system addresses a challenging character-control problem. Swimming requires continuous full-body interaction with fluid, and the combination of reference-guided control and an efficient coupled simulator provides a practical solution. Table 3 shows substantially higher task success than the adapted imitation-learning baselines.

  2. The generalization evaluation is extensive. Training uses a simple still-water task and one reference motion per style, while testing covers unseen trajectories, flows, fluid properties, external forces, and morphology changes. These results support the system’s utility beyond reproducing the reference motion.

Critical weaknesses
  1. The advantage of the main environment representation is not sufficiently established. In Table 4, whole-body raw forces achieve higher success than the complete method, 69.8% versus 63.9%, with favorable position-error and roll metrics. Per-body tokens also achieve higher success. The proposed combination may offer benefits in motion quality, but the reported results do not clearly establish why it is preferable to these simpler alternatives.

  2. The comparisons primarily establish system-level effectiveness. The imitation baselines are adapted to swimming, while the fluid-model comparisons replace the transition model. These comparisons do not fully isolate the contribution of the tokenizer. Moreover, high-resolution SPH and CFC receive matched training time rather than matched sample counts, limiting conclusions about their attainable control quality.

Questions for authors
  1. What measurable advantage justifies combining whole-body raw forces with per-link tokens, given the stronger task metrics of the whole-body raw baseline in Table 4?

  2. Does quantization measurably reduce temporal feature variation or improve training stability compared with a continuous GCN encoder?

  3. Could you clarify how much of the improvement over the imitation baselines comes from the environment representation versus the reference-based action parameterization and reward design?

5 comments
AuthorsOpenReview ↗

Response to Reviewer miux (1/5)

We thank the reviewer for their time and constructive feedback. We appreciate their positive assessment of “SWIM presents a useful system for a challenging swimming-control problem”, and their view that its “system-level contribution and demonstrated capabilities are sufficient to support weak acceptance.” Below, we present the requested additional experiment, address the reviewer’s concerns about our method, and provide the requested clarifications.

1. Additional Experiments

1.1 “The comparisons primarily establish system-level effectiveness. The imitation baselines are adapted to swimming, while the fluid-model comparisons replace the transition model. These comparisons do not fully isolate the contribution of the tokenizer.”

Thank you for this comment. We agree that these comparisons primarily evaluate the complete system. We address the tokeniser’s contribution through the additional representation ablations reported below.

We evaluated 9 additional representation settings, shown in the last nine rows below. All training and evaluation settings follow the paper ablation study. The per-body raw (EMA) inputs use the same log transformation and EMA preprocessing as the tokeniser inputs.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
SWIM(Whole-body raw + Per-body token(EMA α = 0.22))99.7±0.263.9±4.70.211±0.0240.101±0.0240.857±0.0150.94±0.541.31±0.2464.84±14.59
Per-body token96.4±6.868.5±4.60.193±0.0220.101±0.0250.871±0.0152.46±3.281.34±0.2567.23±15.13
Per-body raw(EMA)97.5±4.767.1±5.50.192±0.0210.096±0.0130.880±0.0251.81±2.401.28±0.2263.30±13.84
W/O environment state97.8±2.771.5±4.40.178±0.0290.097±0.0170.901±0.0311.76±2.301.36±0.2467.12±13.22
Whole-body raw + Per-body raw(EMA α = 0.22)99.3±1.266.2±4.80.192±0.0150.100±0.0250.881±0.0091.17±1.211.29±0.2564.47±13.83
Whole-body raw + Per-body raw(EMA α = 0.2)97.3±5.366.6±4.40.198±0.0140.099±0.0250.873±0.0091.85±2.491.28±0.2563.87±13.72
Whole-body raw + Per-body raw(EMA α = 0.1)95.5±8.363.6±6.10.206±0.0180.095±0.0170.866±0.0102.66±3.841.26±0.2662.78±14.99
Whole-body raw + Per-body raw(EMA α = 0.05)96.3±6.569.1±4.30.196±0.0250.098±0.0180.864±0.0112.06±2.941.26±0.2963.59±15.27
SWIM(Whole-body raw + Per-body token(EMA α = 0.2))99.8±0.161.0±2.00.222±0.0200.108±0.0400.860±0.0130.87±0.501.30±0.2364.79±15.37
SWIM(Whole-body raw + Per-body token(EMA α = 0.1))99.5±0.864.2±6.00.207±0.0270.104±0.0260.861±0.0120.91±0.661.28±0.2562.71±14.51
SWIM(Whole-body raw + Per-body token(EMA α = 0.05))99.5±0.666.1±7.50.198±0.0310.097±0.0190.861±0.0240.86±0.331.26±0.2863.58±15.67

Without environment state. This variant achieves the highest mean SR and the best mean PS and Prog. However, it also has the highest mean joint velocity error relative to the reference motion (1.36 ± 0.24 rad/s) and relatively high mean joint acceleration (67.12 ± 13.22 rad/s²). It is ranked least natural overall in the user study below; some participants described its motion as “Actions of someone drowning”. This indicates w/o environment states leads to unnatural motion to achieve better task performance.

Per-body representations. Comparing the two Per-body representations, Per-body token has a slightly higher mean SR than Per-body raw (EMA): 68.5 ± 4.6% versus 67.1 ± 5.5%. However, Per-body raw (EMA) has lower joint angular velocity to reference motion (1.28 ± 0.22 versus 1.34 ± 0.25 rad/s) and joint acceleration (63.30 ± 13.84 versus 67.23 ± 15.13 rad/s²).

AuthorsOpenReview ↗

Response to Reviewer miux (2/5)

1. Additional Experiments

Combined representations and EMA sensitivity. Across the tested EMA parameters, SWIM maintains the best mean SRV of 99.5-99.8%, compared with 95.5-99.3% for Whole-body raw + Per-body raw (EMA). The Whole-body raw + Per-body raw (EMA) variant usually achieves higher mean SR, while both representations have similar Vel and Accel values. In the user study at α = 0.22, matching the setting in the paper, the two methods receive the same overall mean rank.

The following table reports SRV (%) for trajectory following (T2), including interpolation (interp.) and extrapolation (extrap.) settings.

MethodT2T2 interp.T2 extrap.T2 × E1T2 × E1 interp.T2 × E1 extrap.
Whole-body raw + Per-body raw(EMA α = 0.22)100.0±0.0100.0±0.0100.0±0.099.2±1.399.4±0.899.0±2.1
Whole-body raw + Per-body raw(EMA α = 0.2)98.3±3.399.3±1.496.9±6.296.1±7.496.3±6.895.8±8.3
Whole-body raw + Per-body raw(EMA α = 0.1)97.5±5.098.6±2.895.8±8.393.0±11.894.5±9.690.6±15.0
Whole-body raw + Per-body raw(EMA α = 0.05)97.1±4.897.9±2.795.8±8.393.4±11.493.8±10.393.0±13.0
SWIM(Whole-body raw + Per-body token(EMA α = 0.22))100.0±0.0100.0±0.0100.0±0.099.0±0.798.9±1.299.2±0.5
SWIM(Whole-body raw + Per-body token(EMA α = 0.2))100.0±0.0100.0±0.0100.0±0.099.6±0.299.5±0.399.9±0.3
SWIM(Whole-body raw + Per-body token(EMA α = 0.1))100.0±0.0100.0±0.0100.0±0.099.3±0.899.3±1.299.3±0.8
SWIM(Whole-body raw + Per-body token(EMA α = 0.05))100.0±0.0100.0±0.0100.0±0.099.3±0.699.4±0.699.2±0.7

All four SWIM variants achieve 100% SRV in T2 still water condition. The Whole-body raw + Per-body raw (EMA) also achieves 100% at α = 0.22, but its mean SRV decreases at the other tested values. In T2 × E1, SWIM maintains a mean SRV of 99.0-99.6%, while the Whole-body raw + Per-body raw (EMA) ranges from 93.0% to 99.2%. These results suggest that SWIM’s survival performance is less sensitive to the EMA parameter, particularly in trajectory following.

Further analysis shows that the lower SRV of the Whole-body raw + Per-body raw (EMA) representation is mainly associated with backstroke. The table below reports the backstroke results.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
Whole-body raw + Per-body raw(EMA α = 0.22)97.6±4.263.2±7.10.207±0.0470.134±0.0470.891±0.0182.95±3.301.24±0.1257.97±6.16
Whole-body raw + Per-body raw(EMA α = 0.2)89.3±18.461.4±4.60.216±0.0470.133±0.0220.882±0.0215.59±7.741.19±0.0356.49±4.05
Whole-body raw + Per-body raw(EMA α = 0.1)83.1±21.056.6±5.60.233±0.0490.121±0.0100.867±0.0208.37±10.321.12±0.0252.52±2.78
Whole-body raw + Per-body raw(EMA α = 0.05)86.6±23.062.9±6.80.232±0.0560.121±0.0210.859±0.0196.45±8.401.16±0.0455.91±0.90
SWIM(Whole-body raw + Per-body token, EMA α = 0.22)99.7±0.458.8±4.80.225±0.0450.136±0.0600.870±0.0081.70±1.531.24±0.0856.31±6.32
SWIM(Whole-body raw + Per-body token, EMA α = 0.2)99.7±0.458.8±8.80.252±0.1050.168±0.1170.875±0.0101.57±0.351.23±0.0756.66±4.49
SWIM(Whole-body raw + Per-body token, EMA α = 0.1)99.8±0.257.0±2.80.226±0.0080.143±0.0330.873±0.0171.79±0.291.17±0.0754.13±4.20
SWIM(Whole-body raw + Per-body token, EMA α = 0.05)98.7±2.064.2±7.50.198±0.0350.122±0.0420.884±0.0201.09±0.501.14±0.0554.26±2.93

For backstroke, SWIM maintains a mean SRV of 98.7-99.8%, compared with 83.1-97.6% for the Whole-body raw + Per-body raw (EMA) representation. SWIM also has lower SRV variability across seeds. This supports the combined tokenised representation as a more reliable choice for survival across the tested EMA settings.

AuthorsOpenReview ↗

Response to Reviewer miux (3/5)

1. Additional Experiments

As mentioned in Appendix C 1.1, "We do not claim that this is the optimal environment state in all metrics. Instead, our design is motivated by the trade-off between force-estimation accuracy and learning efficiency … Within the space of structured, low-dimensional representations, the proposed combination provides a practical balance between generalization performance and motion quality."

We thank the reviewer for this suggestion. Our original study did not cover all representation choices and missed Per-body raw (EMA) and Whole-body raw + Per-body raw (EMA). These additional experiments help clarify the trade-offs between the representations.

Our contribution includes a low-dimensional state representation that effectively distills complex character-fluid interactions; we also do not claim that SWIM is the best across all metrics. Having said it, based on these results of new representations, we do admit that the ones with Per-body raw (EMA) could give good results sometimes. Therefore, we are open to revising the paper to include EMA-based and tokenised variants of SWIM. In addition, we would like to mention that tokenization may also offer opportunities for future research where transformers can be used, such as combining fluid representations with language embeddings, although we have not evaluated this possibility here. We welcome the reviewer’s suggestions.

Since we cannot include videos in the rebuttal, we conducted a user study to assess perceived motion quality. The lower the score, the better.

StrokeSWIMW/O environment stateWhole-body raw + Per-body raw(EMA)
All strokes1.92±0.352.17±0.281.92±0.30
Front crawl2.00±0.532.25±0.531.75±0.61
Backstroke2.08±0.691.73±0.632.19±0.66
Breaststroke1.65±0.542.60±0.441.75±0.53
Butterfly1.94±0.402.08±0.481.98±0.45

The user study collects results of 24 participants, of 91.7% reported experience in recreational or competitive swimming, coaching, or lifeguarding. Each participant evaluated 8 side view video clips covering four strokes and two goal settings per stroke (a large goal angle and a small goal angle), the same task setting in the supplementary webpage ablation study. The participant was shown a series of video clips, each of which contains the results of three methods mentioned in the table above. For each video clip, participants ranked the three methods from 1 (most natural) to 3 (least natural). The methods are unlabelled, and their order was shuffled for each question. For each stroke, all three methods use the same training seed as the supplemental material videos. The seed selected by the SWIM's best task performance instead of motion quality.

Overall, SWIM and Whole-body raw + Per-body raw (EMA) receive the same mean rank of 1.92, while W/O environment state receives a worse mean rank 2.17. This indicates W/O environment state achieves unnatural controlled motion.

Compared with Whole-body raw + Per-body raw (EMA), SWIM has a better mean rank for backstroke (2.08 versus 2.19), breaststroke (1.65 versus 1.75), and butterfly (1.94 versus 1.98), but a worse mean rank for front crawl (2.00 versus 1.75). W/O environment state receives the best mean rank for backstroke, despite being ranked least natural overall.

Reviewer_k6iJOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SWIM, a reinforcement learning framework for controlling full-body human swimming within a GPU-accelerated coupled rigid–fluid simulation. The policy combines whole-body forces and torques with per-link embeddings produced by a dual-branch graph-convolutional VQ-VAE, while adjusting the reference joint poses and stroke frequency to shape the resulting motion. Using one reference motion per style, the authors train a separate policy for each of four swimming strokes and evaluate task performance and motion quality through baseline comparisons, ablations, and tests under unseen goals, trajectories, fluid conditions, perturbations, and body modifications.

Critical strengths
  1. A practical system for a challenging control problem. SWIM addresses the difficult task of controlling a simulated humanoid swimmer through full-body interaction with fluid. It combines force observations, a reference-motion prior, and coupled simulation, training a separate policy for each stroke from a single reference motion. The GPU implementation reaches a reported throughput of 1,023 samples/s, compared with 65 samples/s for the CPU version on the same workstation. This acceleration makes coupled fluid simulation more practical for policy learning.

  2. Thorough evaluation of capabilities and limits. The reported success rate of 63.9% for SWIM, versus 36.8% for the adapted AMP baseline under the same 10M-sample budget, supports the effectiveness of the full system. Evaluation spans four swimming styles and tests generalization to unseen trajectories, fluid conditions, external perturbations, and body modifications including limb removal and fins. Quantitative failure analysis characterizes performance boundaries across these conditions. Supplementary materials provide baseline comparisons and representation ablations, while demonstrations tracing the letters I/C/L/R further illustrate the system’s ability to follow complex paths.

Critical weaknesses
  1. The benefit of the tokenized representation is not sufficiently isolated. Appendix D.3 states that VQ inputs receive temporal EMA smoothing, whereas raw force observations are unsmoothed. The raw-versus-token comparisons therefore conflate two changes: representation and temporal preprocessing. Appendix B.3 provides useful reconstruction ablations that support the encoder design, but reconstruction quality alone does not establish a downstream control benefit. A controlled comparison would hold force information, preprocessing, policy architecture, and training budget constant, varying only the representation.

  2. The motion-quality advantage over a simpler representation remains unclear. In Table 4, Whole-body raw outperforms SWIM on core task metrics: 69.8% versus 63.9% success rate and 0.177 m versus 0.211 m position error. SWIM shows modestly lower mean Vel and Accel scores, yet its Roll error is higher (0.94 versus 0.74). The supplementary videos are helpful, but the paper does not clearly establish which motion features improve, in which strokes or tasks, or how consistently across training seeds. Interpreting the reported variability is further complicated by conflicting descriptions: Appendix A.7 defines standard deviations over training seeds, while the Table 14 caption describes variation across swimming strokes and refers to Table 4. These gaps weaken confidence in the claimed trade-off between task performance and motion quality.

Questions for authors
  1. Contribution of the representation. Could you provide a focused downstream comparison that retains SWIM’s whole-body wrench branch but replaces its per-body tokens with raw forces and torques, using the same EMA preprocessing? Matching the policy architecture, action parameterization, and training budget would help isolate the representation’s contribution to task performance and motion quality.

  2. Motion-quality benefits and consistency. Using the existing ablation videos and results, could you identify the specific motion-quality gains over Whole-body raw and indicate whether they hold across strokes, tasks, and training seeds? Please also reconcile the descriptions of standard deviations in Appendix A.7 and the Table 14 caption, including how the variability in Table 4 is computed.

  3. Training and demonstration protocols. Please clarify how the intermediate policies used to collect tokenizer data were obtained, and whether the imitation baselines share SWIM’s force observations and action parameterization. For the demonstrations tracing the letters I/C/L/R, did the additional 5M training samples for backstroke and breaststroke use the original still-water goal-reaching task or introduce new trajectories or environmental conditions? Were the goal updates and target speed the same as in the quantitative evaluation?

With clearer evidence of the representation’s practical benefits and clarification of these protocols, I would be open to raising my scoring accordingly.

5 comments
AuthorsOpenReview ↗

Response to Reviewer k6iJ (1/5)

We thank the reviewer for their time and constructive feedback. We appreciate their recognition of SWIM as “a practical system for a challenging control problem” and their positive comments on our thorough evaluation of its capabilities and limitations. Below, we present the requested additional experiment, address the reviewer’s concerns about our method, and provide the requested clarifications.

1. Additional experiments

1.1 “Contribution of the representation. Could you provide a focused downstream comparison that retains SWIM’s whole-body wrench branch but replaces its per-body tokens with raw forces and torques, using the same EMA preprocessing? Matching the policy architecture, action parameterization, and training budget would help isolate the representation’s contribution to task performance and motion quality”

Thanks for the suggestion. We evaluated 9 additional representation settings, shown in the last nine rows below. All training and evaluation settings follow the paper ablation study. The per-body raw (EMA) inputs use the same log transformation and EMA preprocessing as the tokeniser inputs.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
SWIM(Whole-body raw + Per-body token(EMA α = 0.22))99.7±0.263.9±4.70.211±0.0240.101±0.0240.857±0.0150.94±0.541.31±0.2464.84±14.59
Per-body token96.4±6.868.5±4.60.193±0.0220.101±0.0250.871±0.0152.46±3.281.34±0.2567.23±15.13
Per-body raw(EMA)97.5±4.767.1±5.50.192±0.0210.096±0.0130.880±0.0251.81±2.401.28±0.2263.30±13.84
W/O environment state97.8±2.771.5±4.40.178±0.0290.097±0.0170.901±0.0311.76±2.301.36±0.2467.12±13.22
Whole-body raw + Per-body raw(EMA α = 0.22)99.3±1.266.2±4.80.192±0.0150.100±0.0250.881±0.0091.17±1.211.29±0.2564.47±13.83
Whole-body raw + Per-body raw(EMA α = 0.2)97.3±5.366.6±4.40.198±0.0140.099±0.0250.873±0.0091.85±2.491.28±0.2563.87±13.72
Whole-body raw + Per-body raw(EMA α = 0.1)95.5±8.363.6±6.10.206±0.0180.095±0.0170.866±0.0102.66±3.841.26±0.2662.78±14.99
Whole-body raw + Per-body raw(EMA α = 0.05)96.3±6.569.1±4.30.196±0.0250.098±0.0180.864±0.0112.06±2.941.26±0.2963.59±15.27
SWIM(Whole-body raw + Per-body token(EMA α = 0.2))99.8±0.161.0±2.00.222±0.0200.108±0.0400.860±0.0130.87±0.501.30±0.2364.79±15.37
SWIM(Whole-body raw + Per-body token(EMA α = 0.1))99.5±0.864.2±6.00.207±0.0270.104±0.0260.861±0.0120.91±0.661.28±0.2562.71±14.51
SWIM(Whole-body raw + Per-body token(EMA α = 0.05))99.5±0.666.1±7.50.198±0.0310.097±0.0190.861±0.0240.86±0.331.26±0.2863.58±15.67

Without environment state. This variant achieves the highest mean SR and the best mean PS and Prog. However, it also has the highest mean joint velocity error relative to the reference motion (1.36 ± 0.24 rad/s) and relatively high mean joint acceleration (67.12 ± 13.22 rad/s²). It is ranked least natural overall in the user study below; some participants described its motion as “Actions of someone drowning”. This indicates w/o environment states leads to unnatural motion to achieve better task performance.

Per-body representations. Comparing the two Per-body representations, Per-body token has a slightly higher mean SR than Per-body raw (EMA): 68.5 ± 4.6% versus 67.1 ± 5.5%. However, Per-body raw (EMA) has lower joint angular velocity to reference motion (1.28 ± 0.22 versus 1.34 ± 0.25 rad/s) and joint acceleration (63.30 ± 13.84 versus 67.23 ± 15.13 rad/s²).

AuthorsOpenReview ↗

Response to Reviewer k6iJ (2/5)

1. Additional experiments

Combined representations and EMA sensitivity. Across the tested EMA parameters, SWIM maintains the best mean SRV of 99.5-99.8%, compared with 95.5-99.3% for Whole-body raw + Per-body raw (EMA). The Whole-body raw + Per-body raw (EMA) variant usually achieves higher mean SR, while both representations have similar Vel and Accel values. In the user study at α = 0.22, matching the setting in the paper, the two methods receive the same overall mean rank.

The following table reports SRV (%) for trajectory following (T2), including interpolation (interp.) and extrapolation (extrap.) settings.

MethodT2T2 interp.T2 extrap.T2 × E1T2 × E1 interp.T2 × E1 extrap.
Whole-body raw + Per-body raw(EMA α = 0.22)100.0±0.0100.0±0.0100.0±0.099.2±1.399.4±0.899.0±2.1
Whole-body raw + Per-body raw(EMA α = 0.2)98.3±3.399.3±1.496.9±6.296.1±7.496.3±6.895.8±8.3
Whole-body raw + Per-body raw(EMA α = 0.1)97.5±5.098.6±2.895.8±8.393.0±11.894.5±9.690.6±15.0
Whole-body raw + Per-body raw(EMA α = 0.05)97.1±4.897.9±2.795.8±8.393.4±11.493.8±10.393.0±13.0
SWIM(Whole-body raw + Per-body token(EMA α = 0.22))100.0±0.0100.0±0.0100.0±0.099.0±0.798.9±1.299.2±0.5
SWIM(Whole-body raw + Per-body token(EMA α = 0.2))100.0±0.0100.0±0.0100.0±0.099.6±0.299.5±0.399.9±0.3
SWIM(Whole-body raw + Per-body token(EMA α = 0.1))100.0±0.0100.0±0.0100.0±0.099.3±0.899.3±1.299.3±0.8
SWIM(Whole-body raw + Per-body token(EMA α = 0.05))100.0±0.0100.0±0.0100.0±0.099.3±0.699.4±0.699.2±0.7

All four SWIM variants achieve 100% SRV in T2 still water condition. The Whole-body raw + Per-body raw (EMA) also achieves 100% at α = 0.22, but its mean SRV decreases at the other tested values. In T2 × E1, SWIM maintains a mean SRV of 99.0-99.6%, while the Whole-body raw + Per-body raw (EMA) ranges from 93.0% to 99.2%. These results suggest that SWIM’s survival performance is less sensitive to the EMA parameter, particularly in trajectory following.

Further analysis shows that the lower SRV of the Whole-body raw + Per-body raw (EMA) representation is mainly associated with backstroke. The table below reports the backstroke results.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
Whole-body raw + Per-body raw(EMA α = 0.22)97.6±4.263.2±7.10.207±0.0470.134±0.0470.891±0.0182.95±3.301.24±0.1257.97±6.16
Whole-body raw + Per-body raw(EMA α = 0.2)89.3±18.461.4±4.60.216±0.0470.133±0.0220.882±0.0215.59±7.741.19±0.0356.49±4.05
Whole-body raw + Per-body raw(EMA α = 0.1)83.1±21.056.6±5.60.233±0.0490.121±0.0100.867±0.0208.37±10.321.12±0.0252.52±2.78
Whole-body raw + Per-body raw(EMA α = 0.05)86.6±23.062.9±6.80.232±0.0560.121±0.0210.859±0.0196.45±8.401.16±0.0455.91±0.90
SWIM(Whole-body raw + Per-body token, EMA α = 0.22)99.7±0.458.8±4.80.225±0.0450.136±0.0600.870±0.0081.70±1.531.24±0.0856.31±6.32
SWIM(Whole-body raw + Per-body token, EMA α = 0.2)99.7±0.458.8±8.80.252±0.1050.168±0.1170.875±0.0101.57±0.351.23±0.0756.66±4.49
SWIM(Whole-body raw + Per-body token, EMA α = 0.1)99.8±0.257.0±2.80.226±0.0080.143±0.0330.873±0.0171.79±0.291.17±0.0754.13±4.20
SWIM(Whole-body raw + Per-body token, EMA α = 0.05)98.7±2.064.2±7.50.198±0.0350.122±0.0420.884±0.0201.09±0.501.14±0.0554.26±2.93

For backstroke, SWIM maintains a mean SRV of 98.7-99.8%, compared with 83.1-97.6% for the Whole-body raw + Per-body raw (EMA) representation. SWIM also has lower SRV variability across seeds. This supports the combined tokenised representation as a more reliable choice for survival across the tested EMA settings.

As mentioned in Appendix C 1.1, "We do not claim that this is the optimal environment state in all metrics. Instead, our design is motivated by the trade-off between force-estimation accuracy and learning efficiency … Within the space of structured, low-dimensional representations, the proposed combination provides a practical balance between generalization performance and motion quality."

We thank the reviewer for this suggestion. Our original study did not cover all representation choices and missed Per-body raw (EMA) and Whole-body raw + Per-body raw (EMA). These additional experiments help clarify the trade-offs between the representations.

AuthorsOpenReview ↗

Response to Reviewer k6iJ (3/5)

1. Additional experiments

Our contribution includes a low-dimensional state representation that effectively distills complex character-fluid interactions; we also do not claim that SWIM is the best across all metrics. Having said it, based on these results of new representations, we do admit that the ones with Per-body raw (EMA) could give good results sometimes. Therefore, we are open to revising the paper to include EMA-based and tokenised variants of SWIM. In addition, we would like to mention that tokenization may also offer opportunities for future research where transformers can be used, such as combining fluid representations with language embeddings, although we have not evaluated this possibility here. We welcome the reviewer’s suggestions.

Since we cannot include videos in the rebuttal, we conducted a user study to assess perceived motion quality. The lower the score, the better.

StrokeSWIMW/O environment stateWhole-body raw + Per-body raw(EMA)
All strokes1.92±0.352.17±0.281.92±0.30
Front crawl2.00±0.532.25±0.531.75±0.61
Backstroke2.08±0.691.73±0.632.19±0.66
Breaststroke1.65±0.542.60±0.441.75±0.53
Butterfly1.94±0.402.08±0.481.98±0.45

The user study collects results of 24 participants, of 91.7% reported experience in recreational or competitive swimming, coaching, or lifeguarding. Each participant evaluated 8 side view video clips covering four strokes and two goal settings per stroke (a large goal angle and a small goal angle), the same task setting in the supplementary webpage ablation study. The participant was shown a series of video clips, each of which contains the results of three methods mentioned in the table above. For each video clip, participants ranked the three methods from 1 (most natural) to 3 (least natural). The methods are unlabelled, and their order was shuffled for each question. For each stroke, all three methods use the same training seed as the supplemental material videos. The seed selected by the SWIM's best task performance instead of motion quality.

Overall, SWIM and Whole-body raw + Per-body raw (EMA) receive the same mean rank of 1.92, while W/O environment state receives a worse mean rank 2.17. This indicates W/O environment state achieves unnatural controlled motion.

Compared with Whole-body raw + Per-body raw (EMA), SWIM has a better mean rank for backstroke (2.08 versus 2.19), breaststroke (1.65 versus 1.75), and butterfly (1.94 versus 1.98), but a worse mean rank for front crawl (2.00 versus 1.75). W/O environment state receives the best mean rank for backstroke, despite being ranked least natural overall.

2. Method concerns

2.1 “The motion-quality advantage over a simpler representation remains unclear. In Table 4, Whole-body raw outperforms SWIM on core task metrics: 69.8% versus 63.9% success rate and 0.177 m versus 0.211 m position error. SWIM shows modestly lower mean Vel and Accel scores, yet its Roll error is higher (0.94 versus 0.74)”

Thank you for raising this concern. We conducted a user study comparing SWIM, Whole-body raw, and Per-body token to assess perceived motion quality. The user study settings are same as above.

The lower the score, the better.

StrokeSWIMWhole-body rawPer-body token
All strokes1.98±0.282.05±0.261.96±0.28
Front crawl1.65±0.482.81±0.321.54±0.53
Backstroke2.38±0.491.40±0.442.23±0.66
Breaststroke1.73±0.492.08±0.562.19±0.44
Butterfly2.19±0.441.92±0.521.90±0.64

Overall, Per-body token receives a slightly better mean rank than SWIM ( 1.96 ± 0.28 vs. 1.98 ± 0.28), while Whole-body raw receives a slightly worse mean rank (2.05 ± 0.26). So, in general, receives a slightly better overall mean rank than Whole-body, which is the simpler representation.

Reviewer_qMotOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper learns simulated humanoid swimming with RL. The authors reimplement the DFSPH fluid simulation with MuJoCo Warp and train a policy to learn a single reference motion for swimming strokes. The policy is trained in a small pool with still water and then tested zero-shot on new goals and trajectories, flows, fluids, perturbations, fins and removed limbs. It is better than the best imitation learning baseline and the paper shows that policies that are trained with other fluid models mostly fail.

Critical strengths

The setting is pretty interesting and what the authors propose seems to work pretty well. To my knowledge this is the first RL policy for humanoid swimming like that, and the zero-shot tests are really extensive and good (flows, five real fluids, impulses, fins, removed limbs, etc.). Also the GPU DFSPH seems useful for anyone who wants to do RL with fluids, I suppose.

Critical weaknesses

One of the main contributions is the compact tokenized environment state, but it is not really well supported by the paper's own ablations. In Table 4 the simplest variant, "Whole-body raw", has the highest SR, the lowest PS and LD and the best Prog and Roll, and "Per-body token" alone also has a higher SR than SWIM. So these simple approaches and naive ways of handling the state are at minimum on par with what the authors present. That SWIM moves more naturally is not visible to me in the numbers. Also, the tokens have 78 dimensions, exactly the size of the raw per-link wrench, and only the token inputs are EMA-smoothed while the raw inputs are not, so the effect of the quantization is fully clear. The most basic baseline, no environment state at all (proprioception and goal, maybe with history), is missing as well. Also, a bit minor, "generalizes to ... all four competitive swimming styles" in the abstract is kinda misleading, since every stroke has its own policy.

Questions for authors

What is the performance without any environment state, and with per-body raw forces that get the same log and EMA preprocessing as the tokens?

Do the imitation baselines use the same action space and the same environment state as SWIM?

Is the VQ-VAE trained on front crawl only and then used for all strokes, and which policies produced the rollout data for training it?

4 comments
AuthorsOpenReview ↗

Response to Reviewer qMot (1/4)

We thank the reviewer for their time and constructive feedback. We appreciate their comments that SWIM’s “setting is pretty interesting” and SWIM’s simulator is “useful for anyone who wants to do RL with fluids”, and their recognition that SWIM “is the first RL policy for humanoid swimming …”. Below, we present the requested additional experiment, address the reviewer’s concerns about our method, and provide the requested clarifications.

1. Additional experiments

1.1 “What is the performance without any environment state, and with per-body raw forces that get the same log and EMA preprocessing as the tokens?”

Thanks for the suggestion. We evaluated 9 additional representation settings, shown in the last nine rows below. All training and evaluation settings follow the paper ablation study. The per-body raw (EMA) inputs use the same log transformation and EMA preprocessing as the tokeniser inputs.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
SWIM(Whole-body raw + Per-body token(EMA α = 0.22))99.7±0.263.9±4.70.211±0.0240.101±0.0240.857±0.0150.94±0.541.31±0.2464.84±14.59
Per-body token96.4±6.868.5±4.60.193±0.0220.101±0.0250.871±0.0152.46±3.281.34±0.2567.23±15.13
Per-body raw(EMA)97.5±4.767.1±5.50.192±0.0210.096±0.0130.880±0.0251.81±2.401.28±0.2263.30±13.84
W/O environment state97.8±2.771.5±4.40.178±0.0290.097±0.0170.901±0.0311.76±2.301.36±0.2467.12±13.22
Whole-body raw + Per-body raw(EMA α = 0.22)99.3±1.266.2±4.80.192±0.0150.100±0.0250.881±0.0091.17±1.211.29±0.2564.47±13.83
Whole-body raw + Per-body raw(EMA α = 0.2)97.3±5.366.6±4.40.198±0.0140.099±0.0250.873±0.0091.85±2.491.28±0.2563.87±13.72
Whole-body raw + Per-body raw(EMA α = 0.1)95.5±8.363.6±6.10.206±0.0180.095±0.0170.866±0.0102.66±3.841.26±0.2662.78±14.99
Whole-body raw + Per-body raw(EMA α = 0.05)96.3±6.569.1±4.30.196±0.0250.098±0.0180.864±0.0112.06±2.941.26±0.2963.59±15.27
SWIM(Whole-body raw + Per-body token(EMA α = 0.2))99.8±0.161.0±2.00.222±0.0200.108±0.0400.860±0.0130.87±0.501.30±0.2364.79±15.37
SWIM(Whole-body raw + Per-body token(EMA α = 0.1))99.5±0.864.2±6.00.207±0.0270.104±0.0260.861±0.0120.91±0.661.28±0.2562.71±14.51
SWIM(Whole-body raw + Per-body token(EMA α = 0.05))99.5±0.666.1±7.50.198±0.0310.097±0.0190.861±0.0240.86±0.331.26±0.2863.58±15.67

Without environment state. This variant achieves the highest mean SR and the best mean PS and Prog. However, it also has the highest mean joint velocity error relative to the reference motion (1.36 ± 0.24 rad/s) and relatively high mean joint acceleration (67.12 ± 13.22 rad/s²). It is ranked least natural overall in the user study below; some participants described its motion as “Actions of someone drowning”. This indicates w/o environment states leads to unnatural motion to achieve better task performance.

Per-body representations. Comparing the two Per-body representations, Per-body token has a slightly higher mean SR than Per-body raw (EMA): 68.5 ± 4.6% versus 67.1 ± 5.5%. However, Per-body raw (EMA) has lower joint angular velocity to reference motion (1.28 ± 0.22 versus 1.34 ± 0.25 rad/s) and joint acceleration (63.30 ± 13.84 versus 67.23 ± 15.13 rad/s²).

AuthorsOpenReview ↗

Response to Reviewer qMot (2/4)

1. Additional experiments

Combined representations and EMA sensitivity. Across the tested EMA parameters, SWIM maintains the best mean SRV of 99.5-99.8%, compared with 95.5-99.3% for Whole-body raw + Per-body raw (EMA). The Whole-body raw + Per-body raw (EMA) variant usually achieves higher mean SR, while both representations have similar Vel and Accel values. In the user study at α = 0.22, matching the setting in the paper, the two methods receive the same overall mean rank.

The following table reports SRV (%) for trajectory following (T2), including interpolation (interp.) and extrapolation (extrap.) settings.

MethodT2T2 interp.T2 extrap.T2 × E1T2 × E1 interp.T2 × E1 extrap.
Whole-body raw + Per-body raw(EMA α = 0.22)100.0±0.0100.0±0.0100.0±0.099.2±1.399.4±0.899.0±2.1
Whole-body raw + Per-body raw(EMA α = 0.2)98.3±3.399.3±1.496.9±6.296.1±7.496.3±6.895.8±8.3
Whole-body raw + Per-body raw(EMA α = 0.1)97.5±5.098.6±2.895.8±8.393.0±11.894.5±9.690.6±15.0
Whole-body raw + Per-body raw(EMA α = 0.05)97.1±4.897.9±2.795.8±8.393.4±11.493.8±10.393.0±13.0
SWIM(Whole-body raw + Per-body token(EMA α = 0.22))100.0±0.0100.0±0.0100.0±0.099.0±0.798.9±1.299.2±0.5
SWIM(Whole-body raw + Per-body token(EMA α = 0.2))100.0±0.0100.0±0.0100.0±0.099.6±0.299.5±0.399.9±0.3
SWIM(Whole-body raw + Per-body token(EMA α = 0.1))100.0±0.0100.0±0.0100.0±0.099.3±0.899.3±1.299.3±0.8
SWIM(Whole-body raw + Per-body token(EMA α = 0.05))100.0±0.0100.0±0.0100.0±0.099.3±0.699.4±0.699.2±0.7

All four SWIM variants achieve 100% SRV in T2 still water condition. The Whole-body raw + Per-body raw (EMA) also achieves 100% at α = 0.22, but its mean SRV decreases at the other tested values. In T2 × E1, SWIM maintains a mean SRV of 99.0-99.6%, while the Whole-body raw + Per-body raw (EMA) ranges from 93.0% to 99.2%. These results suggest that SWIM’s survival performance is less sensitive to the EMA parameter, particularly in trajectory following.

Further analysis shows that the lower SRV of the Whole-body raw + Per-body raw (EMA) representation is mainly associated with backstroke. The table below reports the backstroke results.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
Whole-body raw + Per-body raw(EMA α = 0.22)97.6±4.263.2±7.10.207±0.0470.134±0.0470.891±0.0182.95±3.301.24±0.1257.97±6.16
Whole-body raw + Per-body raw(EMA α = 0.2)89.3±18.461.4±4.60.216±0.0470.133±0.0220.882±0.0215.59±7.741.19±0.0356.49±4.05
Whole-body raw + Per-body raw(EMA α = 0.1)83.1±21.056.6±5.60.233±0.0490.121±0.0100.867±0.0208.37±10.321.12±0.0252.52±2.78
Whole-body raw + Per-body raw(EMA α = 0.05)86.6±23.062.9±6.80.232±0.0560.121±0.0210.859±0.0196.45±8.401.16±0.0455.91±0.90
SWIM(Whole-body raw + Per-body token, EMA α = 0.22)99.7±0.458.8±4.80.225±0.0450.136±0.0600.870±0.0081.70±1.531.24±0.0856.31±6.32
SWIM(Whole-body raw + Per-body token, EMA α = 0.2)99.7±0.458.8±8.80.252±0.1050.168±0.1170.875±0.0101.57±0.351.23±0.0756.66±4.49
SWIM(Whole-body raw + Per-body token, EMA α = 0.1)99.8±0.257.0±2.80.226±0.0080.143±0.0330.873±0.0171.79±0.291.17±0.0754.13±4.20
SWIM(Whole-body raw + Per-body token, EMA α = 0.05)98.7±2.064.2±7.50.198±0.0350.122±0.0420.884±0.0201.09±0.501.14±0.0554.26±2.93

For backstroke, SWIM maintains a mean SRV of 98.7-99.8%, compared with 83.1-97.6% for the Whole-body raw + Per-body raw (EMA) representation. SWIM also has lower SRV variability across seeds. This supports the combined tokenised representation as a more reliable choice for survival across the tested EMA settings.

As mentioned in Appendix C 1.1, "We do not claim that this is the optimal environment state in all metrics. Instead, our design is motivated by the trade-off between force-estimation accuracy and learning efficiency … Within the space of structured, low-dimensional representations, the proposed combination provides a practical balance between generalization performance and motion quality."

AuthorsOpenReview ↗

Response to Reviewer qMot (3/4)

1. Additional experiments

We thank the reviewer for this suggestion. Our original study did not cover all representation choices and missed Per-body raw (EMA) and Whole-body raw + Per-body raw (EMA). These additional experiments help clarify the trade-offs between the representations.

Our contribution includes a low-dimensional state representation that effectively distills complex character-fluid interactions; we also do not claim that SWIM is the best across all metrics. Having said it, based on these results of new representations, we do admit that the ones with Per-body raw (EMA) could give good results sometimes. Therefore, we are open to revising the paper to include EMA-based and tokenised variants of SWIM. In addition, we would like to mention that tokenization may also offer opportunities for future research where transformers can be used, such as combining fluid representations with language embeddings, although we have not evaluated this possibility here. We welcome the reviewer’s suggestions.

Since we cannot include videos in the rebuttal, we conducted a user study to assess perceived motion quality. The lower the score, the better.

StrokeSWIMW/O environment stateWhole-body raw + Per-body raw(EMA)
All strokes1.92±0.352.17±0.281.92±0.30
Front crawl2.00±0.532.25±0.531.75±0.61
Backstroke2.08±0.691.73±0.632.19±0.66
Breaststroke1.65±0.542.60±0.441.75±0.53
Butterfly1.94±0.402.08±0.481.98±0.45

The user study collects results of 24 participants, of 91.7% reported experience in recreational or competitive swimming, coaching, or lifeguarding. Each participant evaluated 8 side view video clips covering four strokes and two goal settings per stroke (a large goal angle and a small goal angle), the same task setting in the supplementary webpage ablation study. The participant was shown a series of video clips, each of which contains the results of three methods mentioned in the table above. For each video clip, participants ranked the three methods from 1 (most natural) to 3 (least natural). The methods are unlabelled, and their order was shuffled for each question. For each stroke, all three methods use the same training seed as the supplemental material videos. The seed selected by the SWIM's best task performance instead of motion quality.

Overall, SWIM and Whole-body raw + Per-body raw (EMA) receive the same mean rank of 1.92, while W/O environment state receives a worse mean rank 2.17. This indicates W/O environment state achieves unnatural controlled motion.

Compared with Whole-body raw + Per-body raw (EMA), SWIM has a better mean rank for backstroke (2.08 versus 2.19), breaststroke (1.65 versus 1.75), and butterfly (1.94 versus 1.98), but a worse mean rank for front crawl (2.00 versus 1.75). W/O environment state receives the best mean rank for backstroke, despite being ranked least natural overall.

2. Method concerns

2.1 “That SWIM moves more naturally is not visible to me in the numbers”

Thank you for raising this concern. In the paper ablation study, SWIM achieves the lowest mean Accel (64.84 ± 14.59 rad/s²), compared with Per-body token (67.23 ± 15.13 rad/s²) and Whole-body raw (67.82 ± 12.60 rad/s²). It also achieves the second rank for Vel and Roll.

These two metrics measure different aspects of motion: Accel measures overall joint acceleration, while Vel measures the difference between the controller joint velocities and those in the reference motion. Lower acceleration indicates smoother motion, while lower Vel indicates closer joint velocities with the reference velocities.

We do not claim SWIM moves the mostly naturally in all methods. As mentioned in Appendix C 1.1 "We do not claim that this is the optimal environment state in all metrics. Instead, our design is motivated by the trade-off between force-estimation accuracy and learning efficiency … Within the space of structured, low-dimensional representations, the proposed combination provides a practical balance between generalization performance and motion quality." Our claim is that SWIM provides “a balance between task and motion quality.” in paper Section 4.4. We will clarify this.

AuthorsOpenReview ↗

Response to Reviewer qMot (4/4)

3. Clarifications

3.1 “Also, a bit minor, ‘generalizes to ... all four competitive swimming styles’ in the abstract is kinda misleading.”

Thank you for pointing this out. We will revise the abstract to state that we “evaluate SWIM on all four competitive swimming styles” to avoid implying generalization to unseen strokes.

3.2 “Do the imitation baselines use the same action space and the same environment state as SWIM?”

No. The imitation baselines use their original action spaces and without environment state. This is to ensure minimal changes on these baseline methods for fair comparison, given that none of them was designed for tasks like swimming where the environmental forces to the body are complex. As mentioned in Appendix B.2, “We evaluate whether these methods can transfer to swimming with minimal adaptation, using our goal states, task reward, character, reference motion data, and simulator. This setup requires only minimal adaptation to the swimming task. Without our goal state, these methods cannot determine where to move. Without our task reward, they cannot learn the task objective. Without our simulator, they cannot reproduce swimming, which requires hydrodynamic forces, whereas these methods were originally designed for land-based tasks and use simulators without fluid interactions. Without our reference motion data, they cannot perform human motion imitation. For a fair comparison, we use the same character and training settings as their original methods for all methods”

3.3 “Is the VQ-VAE trained on front crawl only and then used for all strokes, and which policies produced the rollout data for training it?”

In the original experiments, we train a separate VQ-VAE for each stroke and use it for the corresponding swimming policy. Thank you for suggesting the use of a shared tokeniser. We conducted an additional experiment in which we reused the front-crawl tokeniser to train policies for the other three strokes.

MethodSRV↑(%)SR↑(%)PS↓(m)LD↓(m)Prog→1Roll↓(°)Vel↓(rad/s)Accel↓(rad/s²)
SWIM(per-stroke tokenisers)99.7±0.263.9±4.70.211±0.0240.101±0.0240.857±0.0150.94±0.541.31±0.2464.84±14.59
SWIM with the front-crawl tokeniser for all strokes99.7±0.259.3±2.90.218±0.0080.098±0.0230.855±0.0190.89±0.261.30±0.2564.23±16.25

Using the front-crawl tokeniser for all strokes reduces the mean SR from 63.9 ± 4.7% to 59.3 ± 2.9%, while the mean SRV remains at 99.7 ± 0.2%. The other metrics remain numerically close to those of the original SWIM. These results suggest that the front-crawl tokeniser can support policy training for the other strokes, although performance decreases on SR from 63.9 ± 4.7% to 59.3 ± 2.9%.

In the original experiments, each stroke’s tokeniser is trained only on data from that stroke, collected in still water. The data include reference motion playback by PD controller and policy rollouts. The rollout policy is trained for that stroke using per-body raw force observations on the still-water goal-reaching task. We collect rollouts from its final checkpoint using both deterministic and stochastic actions, and from four intermediate checkpoints (0.5M, 2.5M, 4.5M and 6.5M samples). The each stroke dataset contains around 450 episodes, in total around 175k frames.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.