acceptodds
Under review as a conference paper at ICLR 2027

PhysMAS: Physics-Grounded Multi-Agent Synthesis of Compositional 4D Gaussians

Abstract

Efficient, fully automatic, and physically plausible 4D Gaussian synthesis is an important goal for dynamic scene generation. Recent physics-based methods couple 3D Gaussians with the Material Point Method (MPM) to generate physically driven motion, but extending this paradigm to heterogeneous multi-part objects and interacting multi-object scenes remains challenging. Object-level physical assignment collapses distinct parts into a single material state, while one-shot predictions from large language models, vision-language models, or agents neither reliably bind different materials to identified parts nor verify that the resulting MPM configuration is executable. Score Distillation Sampling (SDS)-based parameter optimization, meanwhile, requires repeated per-scene score evaluations and gradient backpropagation, incurring lengthy optimization and potentially yielding suboptimal or unstable solutions. We therefore present PhysMAS, a physics-grounded multi-agent framework. From a motion prompt and four scene views, an Object-Part Scene Agent establishes persistent identities and calls a Material Reasoning Agent for part-wise profiles. It invokes solver-aware skills to bind these identities and profiles to per-particle MPM fields and execute all objects in a shared domain; the framework then screens candidate forward-simulation results. This supports heterogeneous multi-part and interacting multi-object scenes without per-scene diffusion-score backpropagation. Extensive experiments demonstrate that, compared with recent physics-based 4D Gaussian baselines that rely on SDS, PhysMAS achieves better semantic alignment and perceived physical plausibility while requiring less runtime.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Physics-Grounded 4D Generation, Gaussian Splatting, Multi-Agent Systems, part-aware physical simulation, Material Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_7LZBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PhysMAS introduces a multi-agent, physics-grounded framework for generating compositional 4D Gaussian scenes. A scene agent identifies and maintains persistent object–part relationships, while a material agent infers part-wise physical properties for MPM simulation. The system supports heterogeneous multi-part objects and interacting multi-object scenes without per-scene SDS optimization. Experiments show that PhysMAS outperforms several existing methods in semantic alignment, user preference, motion consistency, and deformation quality, while ablation studies confirm the importance of object–part modeling and structure-preserving projection.

Critical strengths

This paper addresses the problem that the structures of complex objects cannot be properly separated in current simulator-driven 4D Gaussian synthesis methods, and proposes an agent-based framework to solve it. PhysMAS consists of two agents: one is responsible for identifying complex object–part identities, planning actions, and accepting the final results, while the other determines the physical parameters of each part. Compared with SDS-based methods, PhysMAS achieves substantially faster inference.

Critical weaknesses

The entire system relies on Grounding DINO and SAM 2 to segment object parts. Errors at this stage can propagate to material prediction and subsequent MPM simulation. Does the paper provide any robustness analysis for segmentation errors? Compared with SDS-based methods, PhysMAS may have limited ability to correct such errors once the initial part assignments are incorrect.

The candidate screening process and predefined material categories incorporate substantial human priors. This makes the method appear more like an application of agents to 3D material styling than a fundamentally new physics-based approach. Compared with SDS-based methods, PhysMAS may also exhibit a noticeable gap in physical realism, with its main advantage being efficiency. The qualitative analysis mainly demonstrates relatively simple object transformations, such as rigid objects falling. More complex cases may produce implausible results. For example, in the flowerpot example shown on page 29 of the appendix, the flowerpot falls while the grass remains static, which appears physically unnatural.

Questions for authors

How does the agent handle severe occlusion or incomplete visual evidence? For example, how would it reason about a piece of dough wrapped around or enclosing a stone when the internal structure cannot be directly observed?

Reviewer_zhHQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses physics-based 4D Gaussian synthesis for scenes with multi-part objects (e.g. a cake with plate, base and toppings) and several interacting objects. PhysMAS takes a motion prompt, four calibrated views and a static 3DGS. An object-part scene agent (a VLM plus GroundingDINO/SAM 2) labels parts on the Gaussians and builds an action plan. A material reasoning agent proposes part-wise material evidence and E/nu statistics (using PhysGM's predictor), and hand-written rules turn these into up to six candidate MPM configurations. All objects are simulated in one shared MPM grid, and candidates are screened, ranked and repaired by hand-designed scores over up to 3 rounds. Evaluation compares against OmniPhysGS, PhysSplat, DreamPhysics and PhysGM using CLIP similarity, a 20-person user study, errors against author-made MPM reference videos, runtime, and two ablations.

Critical strengths
  • The evaluation goes beyond CLIP similarity, which is a weak signal for physics. It adds a blinded user study, trajectory, area and strain errors, runtime, and qualitative ablations.
Critical weaknesses
  • Limited significance and contribution. Assigning physical attributes to parts of an object from a text or image description is something a VLM can already do directly, and the pipeline here is mostly a VLM call plus hand-written rules (keyword lookups such as 'frosting/cream -> Foam', 'plate -> Ceramic', many frozen thresholds and weights) on top of an existing MPM simulator and an existing E/nu predictor (PhysGM). The paper does not show what the 'agentic' structure adds beyond this. It reports no one-shot LLM/VLM baseline, even though failure of one-shot guessing is its stated motivation, and no comparison with PhysAgent, which it cites. There are no ablations of the material agent, the screening and repair loop, or multi-candidate search, so the multi-agent claim is not isolated. The problem setting itself (heterogeneous materials per part) is a modest extension of per-object material assignment, and the paper offers no new insight into physics, simulation or learning.

  • The evaluation is weak and may favor the method. (a) The motion and strain errors in Table 3 are measured against reference videos the authors hand-authored with MPM, using the same static 3DGS and renderer, so methods with other simulators are judged against PhysMAS-like ground truth, and there is no real-world or independent reference. (b) The runtime comparison is not meaningful: DreamPhysics and OmniPhysGS optimize materials per scene with SDS, which is a different and much more expensive paradigm, while PhysMAS reuses an off-the-shelf predictor and rules. Being faster than SDS says little about quality or contribution, and the runtime may not include LLM API latency or the up to 3 repair rounds. (c) The number of test scenes, user-study responses and failure cases is not reported, and there are no variances, confidence intervals or significance tests. CLIP-similarity margins over PhysGM are small (e.g. 0.3021 vs 0.2997 on Sand). (d) The hand-tuned rules and thresholds have no stated tuning procedure or held-out split, and the showcased scenes (cake, hammer) match the rules closely, which raises overfitting concerns.

Questions for authors

What do a one-shot VLM/LLM baseline, PhysAgent, and ablations without the material agent, without screening and repair, and with PhysGM per-part prediction alone achieve? This is needed to attribute any gains to the multi-agent design instead of to the rules or the reused predictor.

Reviewer_3iQXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PhysMAS is an automated physics-based 4D Gaussian animation framework for heterogeneous multi-part objects and interacting multi-object scenes. The method eliminates tedious per-scene SDS optimization, employing an "Object-Part Scene Agent" and a "Material Reasoning Agent" combined with deterministic MPM simulations to efficiently handle part identity tracking, material reasoning, and physical simulation and screening. Compared to existing baselines, PhysMAS achieves superior quality metrics across all evaluations while reducing the post-reconstruction runtime for a 50-frame sequence to approximately two minutes.

Critical strengths

What I like most about the system design is how carefully it maintains (object, part) identities across the whole pipeline, from 2D visual grounding and material prediction down to 3D Gaussian labeling and MPM particle initialization. In practice, bridging 3DGS and MPM on multi-part or multi-object scenes is messy because material labels easily bleed across boundaries when you convert surface Gaussians into volumetric particles. The authors handle these boundary cases thoughtfully. For instance, when generating interior filler particles for MPM, each particle is locked to its parent object first before looking up the nearest Gaussian for its part label, which prevents parts from accidentally inheriting materials from a neighboring object. Similarly, keeping object-scoped residual rows ensures every particle still gets a valid fallback material even when 2D masks miss some Gaussians.

This makes for a solid systems contribution. Instead of treating part segmentation and physics simulation as separate problems, the paper gives a clean, working pipeline where heterogeneous parts and interacting objects can actually run together in a shared MPM grid without falling apart. Having clear data structures (like Table A.1) and deterministic compiler checks between the VLM outputs and the solver also makes the whole system much easier to debug and reproduce than end-to-end black-box optimization.

Critical weaknesses

The contribution of the two-agent organization is insufficiently established. The Scene Agent owns scene planning and orchestration, while the Material Agent provides part-level evidence, but candidate generation, repair, scoring, and screening are governed by fixed rules. The current ablation removes both the Object-Part Scene Agent and part-aware state and compilation, so it does not isolate the benefit of the agent decomposition itself. A matched comparison retaining the same part assignments, physical-property predictor, simulator, projection options, and candidate budget would help determine how much the proposed organization contributes beyond a simpler part-aware pipeline. This is consequential because the reported gains could arise largely from finer-grained material assignment and the deterministic simulation heuristics, while the additional value of the two-agent contract remains unclear.

Questions for authors

The "w/o O-P Agent" ablation in Section 4.4 removes the Scene Agent, part-level state, and part-aware compilation all at once. How much of the gain comes from the two-agent setup itself versus simply having part-level material assignment (e.g., compared to a scripted or single-call pipeline using the same segmentation tools, predictor, and compiler)? Also, if you only run the initial posterior-MAP candidate without candidate screening or repair rounds, what are the execution success rate, CLIP_sim, and Table 3 motion/deformation errors?

How many unique 3D assets and scene-prompt pairs are in the multi-part and multi-object test sets, and how much do the six profile columns in Table 1 overlap? Appendices B and C rely on specific keyword lists and fixed thresholds for projection, interface bonding, and candidate acceptance. Were these rules and thresholds tuned on a held-out split, or on the evaluated test scenes? If the test scenes were used during rule design, can you show results on held-out scenes with all rules frozen, along with per-scene variance for the main tables?

When \eta_r > 0, Equations (9) and (10) pull particle positions toward their unrotated frame-0 offsets \delta_{qr}^0 and relax F_q^{n_t} toward I_3, which suppresses legitimate rigid-body rotation. How does this work when a support or rigid part needs to topple or rotate? Please also clarify what fraction of selected rollouts use this projection, how the Table 3 errors look for the "w/o S-P Projection" ablation, and whether the hand-authored MPM reference videos also use this projection operator.

Could you break down the ~2-minute runtime across perception, VLM calls, property prediction, MPM rollouts, and rendering? Please clarify whether the 2 minutes includes all rejected candidate simulations and repair rounds, and report the average number of MPM rollouts executed per scene.

Reviewer_ast2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes PhysMAS, an agentic framework for synthesizing physics-based 4D Gaussian animations from static 3D scenes and text prompts. Its core idea is to use two agents to identify objects and their semantic parts, infer part-wise material properties, and configure an MPM simulator to generate physically plausible motion. The framework further introduces techniques to select simulation results without performing lengthy per-scene SDS optimization. Experiments demonstrate improvements in semantic alignment, perceived physical plausibility, and motion and deformation consistency compared with prior physics-based 4D Gaussian generation methods, while maintaining relatively low runtime.

Critical strengths

The paper presents an automated framework for physics-based 4D Gaussian synthesis that supports interactions between multiple objects. The experimental results are promising, demonstrating improvements in semantic alignment, perceived physical plausibility, and motion and deformation consistency over the evaluated baselines, while maintaining relatively low runtime.

Critical weaknesses

I have the following concerns:

  1. The necessity and advantages of the multi-agent framework are not sufficiently justified. The paper emphasizes its multi-agent design but provides limited insight into why such an architecture is necessary. For example, why not merge the Object-Part Scene Agent and the Material Reasoning Agent into a single agent? This concern is further amplified by the lack of comparisons with more capable general-purpose models paired with advanced harnesses, such as Opus 5.5 with Claude Code or GPT-6 Astra with Codex. Given the strong reasoning and tool-use capabilities of these agents, it remains unclear what additional advantages the proposed multi-agent framework provides and what new methodological insights motivate its design.

  2. The motivation for solver-aware candidate construction is unclear. The authors rely on predefined material-parameter variations and heuristic screening to select simulation results, rather than allowing agents to refine physical parameters based on simulation feedback iteratively. It is unclear why this design is preferable to trial and error driven by the agent. A comparison under matched simulation budgets would help establish whether the proposed strategy offers advantages in quality, reliability, or efficiency.

Overall, I believe that the paper does not sufficiently justify its key design choices or provide the corresponding ablations to demonstrate their advantages. These concerns make it difficult for me to assess the methodological novelty and effectiveness.

Questions for authors
  1. What evidence supports the necessity of the multi-agent architecture? Can the authors provide an ablation comparing the proposed framework with a single-agent approach using the same tools and the same model? In particular, does the multi-agent design still provide meaningful improvements when the single-agent baseline uses a sufficiently capable frontier model like Opus 5.5 or GPT-6 Astra?
  2. Following the previous question, how sensitive is the proposed framework to the choice of foundation model? Can the authors provide ablations across different models, including more advanced frontier models, to assess how performance scales with model capability and the potential performance ceiling of the framework?
  3. What motivates using predefined solver-aware candidate construction and heuristic screening instead of allowing agents to iteratively refine physical parameters based on simulation feedback? Can the authors compare these approaches under matched simulation budgets to demonstrate the advantages of the proposed design?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.