acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Route Optimization for Gaussian Head Avatar Modeling

Abstract

Head avatar modeling requires jointly optimizing multiple objectives with different dominant effects on geometry, appearance, and cross-view consistency. However, their relative effectiveness varies across training states, while existing pipelines typically rely on fixed loss weights or handcrafted stage-wise schedules. A central challenge is therefore to identify which optimization direction is more beneficial at each training state. We propose a counterfactual route optimization framework for Gaussian head avatar modeling, which characterizes state-dependent optimization preference from the realized effects of alternative updates rather than predefined heuristic weighting. Starting from the same training state, we perform short-horizon route-restricted lookahead over geometry, appearance, and joint update routes and evaluate their outcomes under a unified utility. The resulting counterfactual evidence is factorized into a geometry–appearance preference and a residual joint advantage, separately capturing the relative preference between individual update directions and the additional benefit of coordinated optimization. We further amortize this offline evidence into a lightweight controller that directly estimates the current optimization preference and applies bounded modulation to the training objectives during full avatar optimization. Experiments on the NeRSemble dataset validate the effectiveness of the proposed design, consistently outperforming existing methods while preserving clearer local facial structures and finer details.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Head Avatar Modeling, 3D Gaussian Splatting, Counterfactual Learning, Dynamic Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FC1DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an adaptive loss-weighting scheme for hybrid mesh–Gaussian head avatars (MeGA representation). Offline, from checkpoints sampled in the first 10% of training, it applies a one-step update restricted to geometry parameters, to appearance parameters, or to both, measures per-term normalized loss improvements with a penalty for degradation of geometric terms, and converts the three utilities into a geometry–appearance preference (softmax of Qg, Qa) and a residual joint advantage ΔQj = Qj − max(Qg, Qa). A small MLP controller (30-dimensional state, hidden size 64) is trained to predict these quantities from the training state and is then used online, after 20K iterations, to scale geometry-, appearance- and consistency-oriented loss groups within [0.9, 1.1]. Experiments on eight NeRSemble subjects report small average improvements over Fresco and MeGA, plus single-subject ablations of the modulation strategy and of the route-evidence factorization.

Critical strengths
  1. The question is real and the approach is reasonable in outline. Geometry, photometric and consistency objectives interact strongly in hybrid avatar training, and deciding their relative emphasis per state from post-update outcomes rather than from loss magnitudes is a sensible idea. Amortizing the decision into a controller leaves inference untouched (57 ms vs 55 ms per frame).

  2. The controls are thoughtfully chosen. Fixed-GC isolates the new cross-view consistency loss, Mean Mod. tests whether a static re-weighting explains the gain, the factorization is compared against GA-only, Joint-only and Parallel-Route variants, and offline cost is reported (1 h 46 min of label generation and 3 min of controller training per subject).

Critical weaknesses
  1. With a single-step lookahead, the "counterfactual" evidence is, to first order, closely related to the gradient statistics the paper positions itself against, and the residual joint advantage may largely reflect additivity. For one step of any diagonally preconditioned optimizer restricted to a parameter block S, the decrease of the training-batch loss is ≈ η⟨∇_S L, P∇_S L⟩, a preconditioned squared gradient norm of that block. Because Sg and Sa are disjoint, Qj ≈ Qg + Qa to first order, hence ΔQj = Qj − max(Qg, Qa) ≈ min(Qg, Qa) ≥ 0 whenever both routes improve, regardless of any interaction between geometry and appearance; the per-term percentile normalization and the degradation penalty in Eqs. 7–9 do not change this qualitatively. Consequently (a) the contrast drawn in Secs. 1 and 2.3 with gradient- and loss-based weighting methods may be stronger than the m = 1 setting supports; (b) it would help to show that ΔQj is not simply ≈ min(Qg, Qa) (a scatter plot from the logged labels, or labels regenerated with m ≫ 1); and (c) specifying the 30-dimensional state descriptor is important, since if it contains loss values or gradient norms the controller is effectively a learned function of those statistics. Independently of this, the offline intervention (restricting the updated parameter subset, Eq. 6) and the online action (re-weighting loss groups by at most ±10%, Eqs. 15–17) are different operations; an experiment showing that the former predicts the benefit of the latter would considerably strengthen the central mechanism.

  2. The empirical evidence is not yet sufficient to establish the claims, and some effect sizes are hard to reconcile with each other. Average gains over Fresco are 0.10 dB (novel view) and 0.19 dB (self-reenactment) with LPIPS tied (Table 1), over eight subjects, with no seeds or per-subject numbers; the abstract's "consistently outperforming" may be stronger than this margin supports. Both ablations use a single subject (264 and 460), and their effect sizes are large relative to the modulation range: on subject 264 a static re-weighting within ±10% (Mean Mod.) yields +0.35 dB and its temporal variation a further +0.38 / +0.79 dB (Table 2); on subject 460 the factorization alone is worth +1.25 dB over GA-only (Table 4), more than the method's entire average gain over MeGA (+1.11 dB, Table 1). Multi-seed, all-subject ablations would be needed to separate these differences from the run-to-run variance of Gaussian avatar training. Part of the gain over the MeGA baseline (Baseline→Fixed-GC, +0.62 / +0.44 dB) comes from the group-consistency loss, which is not among the listed contributions. The controller is fitted only on states from the first 10% of training yet applied from iteration 20K onward (Appendix A); since the total iteration count is not given, it is unclear how far the controller operates outside its training regime, and a prediction-accuracy-over-time analysis would help. Mean Mod. rules out one static schedule; a progress-only (time-dependent) schedule and standard adaptive weighting baselines under equal compute would be natural additional controls, and it would help to state whether Table 3's 17 h includes the per-subject offline cost.

Questions for authors
  1. Mechanism. From the logged offline labels, what is the relationship between ΔQj and min(Qg, Qa) (scatter/correlation)? Do the labels change qualitatively with m = 5–20 lookahead steps? Please list the 30 state features. Does the parameter-restricted route utility predict the benefit of the loss re-weighting actually used online? For example, generate labels by applying the (sg, sa, sj) perturbations for one step from the same state and compare them with the current labels, or report the correlation between the controller's action and the realized improvement on held-out views.

  2. Controller transfer, cost and reproducibility. How many iterations does a subject's training take, what fraction is covered by the label states, and what is the controller's prediction error in early, middle and late training? Is the controller trained per subject, and does the 17 h in Table 3 include label generation and controller training? Please also specify the assignment of losses to Lg / La / Lc, the group-consistency loss, the percentile normalization, whether optimizer state and RNG are restored after each lookahead, and whether code will be released.

  3. Robustness and controls. Please report per-subject results and standard deviations over at least three seeds for Tables 1, 2 and 4, extend both ablations to all eight subjects, and add equal-compute controls: a progress-only controller, GradNorm or uncertainty weighting, and a periodically re-run online lookahead.

2 comments
AuthorsOpenReview ↗

Responses to Reviewer FC1D (Part1)

Thank you for the detailed and constructive comments. Following your suggestions, we added GradNorm, route modulation without group consistency, multi-step lookahead analysis, controller evaluation at later training stages, the relationship between ΔQj\Delta Q_j and min⁡(Qg,Qa)\min(Q_g,Q_a), and complete per-subject results. We also further clarified several implementation and reproducibility details in the revised manuscript.

1. Single-step route evidence and gradient-based weighting

We added GradNorm as a standard gradient-based adaptive weighting baseline under the same Subject 264 training setting as Ours. GradNorm replaces the route controller and adaptively balances the active geometry-, appearance-, and consistency-oriented objective groups throughout training, using its standard setting (α=1.5\alpha=1.5).

MethodNovel PSNRNovel SSIMNovel LPIPSSelf PSNRSelf SSIMSelf LPIPS
Baseline30.360.9520.06629.600.9480.067
GradNorm30.660.9500.05729.760.9470.064
Ours31.710.9560.05730.980.9540.060

GradNorm provides only a limited improvement over the Baseline, while Ours achieves substantially higher PSNR on both tasks: 31.71 vs. 30.66 dB for novel-view synthesis and 30.98 vs. 29.76 dB for self-reenactment. This indicates that standard gradient-based adaptive weighting and our route-based optimization are not equivalent in practice.

Our method does not directly use loss magnitudes or gradient norms for weighting, but evaluates the realized changes of multiple objectives after restricted updates. We agree that, for a single local update, this evidence is theoretically related to gradient statistics. We have softened the original contrast and now describe route evidence as local optimization evidence based on realized post-update outcomes.

2. Relationship between ΔQj\Delta Q_j and min⁡(Qg,Qa)\min(Q_g,Q_a)

We analyzed all 5,600 offline labels of Subject 264. The Pearson and Spearman correlations between ΔQj\Delta Q_j and min⁡(Qg,Qa)\min(Q_g,Q_a) are 0.841 and 0.856, respectively, with R2=0.707R^2=0.707.

However, among the 5,599 states where both Qg>0Q_g>0 and Qa>0Q_a>0, 1,360 states (24.29%) still have ΔQj<0\Delta Q_j<0. Therefore, the two quantities are strongly correlated but not equivalent.

We have accordingly revised the interpretation of ΔQj\Delta Q_j as the residual advantage or disadvantage of the joint route relative to the stronger individual route, rather than as an independent interaction signal.

3. Is single-step lookahead sufficient?

We further generated route evidence using m=5m=5 and m=10m=10 lookahead steps and compared them with the m=1m=1 teacher used in our method:

ComparisonRoute-ranking agreementGA-pref. agreementPref. PearsonQgQ_g PearsonQjQ_j Pearson
m=1m=1 vs. m=5m=586.7%86.7%0.7480.8070.787
m=1m=1 vs. m=10m=1056.7%56.7%0.5470.6570.688

The m=1m=1 evidence is highly consistent with a moderate short-horizon rollout (m=5m=5), supporting m=1m=1 as a useful short-horizon approximation. As the horizon increases, the parameter trajectories of different routes gradually diverge, making the resulting utilities increasingly path-dependent. We therefore use m=1m=1 as a practical local proxy while avoiding the additional forward/backward/update cost of longer rollouts.

4. Controller consistency at later training stages

We recomputed counterfactual evidence at the middle and late training stages and compared it with the controller predictions:

StageGA-Pref. AgreementPearsonSpearman
Middle (~65K)90%0.6700.557
Late (~120K)86%0.3760.358

GA-Pref. Agreement measures whether the controller and the newly recomputed evidence prefer the same geometry or appearance route. Pearson measures the linear correlation between their preference scores, while Spearman measures the consistency of their ranking. These results show that the geometry–appearance preference learned from early trajectory states remains largely consistent with fresh counterfactual evidence during the middle and late stages where the controller is applied.

5. Offline restricted routes vs. online bounded modulation

The two stages serve different purposes. Offline parameter-restricted routes are used to generate discriminative route-level supervision, while the online controller converts the learned optimization preference into low-cost bounded modulation of the original training objectives. Therefore, we do not require the restricted-route utility and the bounded ±10%\pm10\% online re-weighting to be strictly equivalent at every state. Periodically re-running multiple explicit lookahead routes during online training would reintroduce additional forward/backward/update operations and reduce the computational advantage of the amortized controller. We have clarified this distinction in the revised manuscript.

AuthorsOpenReview ↗

Responses to Reviewer FC1D (Part2)

6. Group consistency and route modulation

The original Fixed-GC control isolates the contribution of the consistency objective:

MethodNovel PSNRNovel SSIMNovel LPIPSSelf PSNRSelf SSIMSelf LPIPS
Baseline30.360.9520.06629.600.9480.067
Fixed-GC30.980.9540.06030.040.9500.064
Route Mod. w/o GC31.420.9530.05830.440.9460.065
Ours31.710.9560.05730.980.9540.060

Fixed-GC improves PSNR over Baseline by 0.62 dB on novel-view synthesis and 0.44 dB on self-reenactment. Route Mod. w/o GC further improves Baseline by 1.06 and 0.84 dB, respectively, and outperforms Fixed-GC on both tasks. Adding GC further improves PSNR from 31.42 to 31.71 dB and from 30.44 to 30.98 dB. Thus, route modulation contributes independently, while group consistency provides complementary gains.

7. Does the [0.9, 1.1] clipping make the controller too weak?

We use this conservative range to avoid excessively perturbing the original optimization dynamics. Although each step is bounded, the effect accumulates during training. On Subject 264, self-reenactment PSNR is 29.60 dB for Baseline, 30.19 dB for Mean Mod., 30.44 dB for Route Modulation w/o GC, and 30.98 dB for Full. Mean Mod. uses fixed average modulation, while Full adapts to each training state. The no-GC result also shows that bounded route modulation alone provides a clear gain. Thus, [0.9, 1.1] is conservative but still effective.

8. 30-dimensional state descriptor

The 30-dimensional descriptor contains four categories:

  1. current loss and regularization values, including photometric, geometry, and consistency-related terms;
  2. objective activation and weighting states;
  3. training-stage indicators;
  4. normalized optimization progress, including epoch, global step, and stage progress.

It contains no gradient norms or other gradient statistics. The controller predicts the route preference provided by offline counterfactual evaluation from the current training state, rather than applying a hand-designed gradient-based weighting rule. The complete 30-dimensional feature specification has been added to the revised supplementary material.

9. Per-subject robustness and component-level ablations

Our main comparison covers 8 NeRSemble subjects, while component ablations use representative subjects. This protocol is also used by several closely related person-specific head-avatar methods: GaussianAvatars uses Subject 304, TensorGA Subject 306, MeGA Subject 306, and Fresco Subject 253.

We follow the same protocol and have added complete per-subject comparisons with MeGA and Fresco to the revised supplementary material. The Fresco comparison is shown below.

Novel-view synthesis

SubjectFresco PSNRSSIMLPIPSOurs PSNRSSIMLPIPS
07430.360.9160.10730.250.9170.105
10428.920.9230.09129.040.9250.089
25334.020.9570.03634.110.9590.033
26430.980.9540.06031.710.9560.057
30233.640.9490.04133.490.9480.042
30427.400.8710.07627.670.8710.075
30633.710.9670.03433.540.9670.035
46035.350.9670.02535.380.9670.025
Avg.31.800.9380.05831.900.9390.058

Self-reenactment

SubjectFresco PSNRSSIMLPIPSOurs PSNRSSIMLPIPS
07427.320.8970.11227.390.8980.112
10427.290.9110.09727.430.9130.094
25332.460.9490.04332.660.9510.041
26430.040.9500.06030.980.9540.060
30232.380.9370.04632.180.9370.047
30427.840.8730.08427.930.8720.083
30632.150.9580.04132.240.9580.039
46032.970.9600.03633.140.9610.035
Avg.30.300.9290.06430.490.9310.063

Ours achieves higher PSNR than Fresco on 5/8 subjects for novel-view synthesis and 7/8 for self-reenactment, showing that the overall gain is not driven by a single identity.

10. Computational cost and reproducibility details

The controller is trained separately for each subject. Offline label generation takes about 1 h 46 min per subject and controller training about 3 min. The 17 h in Table 3 covers only full avatar optimization and excludes these offline costs. The revised supplementary material also provides objective grouping, percentile normalization, controller architecture, and online modulation settings. The controller remains fixed during full optimization, with no additional online lookahead.

Thank you again for the detailed analysis. These comments helped us clarify the interpretation of route evidence and strengthen the experimental validation.

Reviewer_RGduOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores novel optimization schemes for 3DGS-based head avatars. Instead of following the default gradient based optimization, which updates all parameters jointly, the authors propose a split of the optimization parameters into geometry and appearance parameters.

A small MLP is trained to predict the ideal set of updated parameters, based on state features as input. To reduce the training costs, the authors choose to train the MLP-route-controller from checkpoints with the first 10% of training. The predictions of the MLP are mapped to weights for loss groups influencing geometry, appearance, or all parameters, but the weights are clipped to [0.9, 1.1], thus only allowing for small deviations from standard gradient descent.

Ultimately, the method shows small quantitative and qualitative improvements compared to baselines on self- and cross reenactment.

Critical strengths
  • Minor quantitative improvements.
Critical weaknesses
  • Missing comparisons to higher-fidelity-class of models, such as GaussianHeadAvatars (GHA), NPGA (code not released), ScaffoldAvatar (code not released). E.g. ScaffoldAvatar shows in their evaluation, that signficantly outperforms GaussianAvatars. I am aware that the model class is slightly different, since there is a screen-space CNN making the final predictions. However, I strongly believe that this class of models predicts superior image quality, such as captured in the LPIPS metrics.
  • The fact that the MLP is trained only during the beginning of training (first 10%) is extremely suspicious. My intuition would tell me that the focus of training will shift heavily from coars to fine. Thus the optimal route behavior might change signifacantly throughout the training. This is not experimentally validated, nor analyzed and compared. Furthermore, while a single-step lookahead could deny bad updates in some cases, other cases required exiting a local minima. Thus a single-step lookahead on the same batch might not be enough to judge the quality of an update. this is also not ablated or analyzed in any way.
Questions for authors
  • How much effect does the route controller have during training with the chosen clipping weights?
3 comments
AuthorsOpenReview ↗

Responses to Reviewer RGdu

Thank you for the detailed comments. We have revised the manuscript and supplementary material to address the concerns regarding comparison scope, controller generalization, lookahead horizon, and the effect of bounded modulation.

1. Comparison with higher-fidelity models

Gaussian Head Avatar (GHA) was included in the original Table 1 under the same 8-subject evaluation protocol. It achieves 29.84 / 0.895 / 0.083 on novel-view synthesis and 22.34 / 0.851 / 0.145 on self-reenactment, compared with 31.90 / 0.939 / 0.058 and 30.49 / 0.931 / 0.063 for Ours.

As the reviewer also noted, ScaffoldAvatar adopts a different rendering pipeline with an additional screen-space CNN, making a direct numerical comparison less controlled. We therefore do not include potentially biased reproduced results. We have discussed NPGA and ScaffoldAvatar in the revised related work to clarify their differences from our optimization-focused setting.

2. Controller generalization beyond early training

We have added a controller generalization analysis to the revised supplementary material. We recompute fresh counterfactual evidence at middle and late training stages and compare it with the fixed controller predictions:

StageGA-Pref. AgreementPearsonSpearman
Middle (~65K)90%0.6700.557
Late (~120K)86%0.3760.358

GA-Pref. Agreement measures whether the controller and fresh counterfactual evidence prefer the same geometry or appearance route. Although the controller is supervised using early trajectory states, its geometry–appearance preference remains largely consistent with fresh evidence during the middle and late stages where it is actually deployed.

3. Single-step lookahead

We have added a multi-step lookahead analysis in the revised supplementary material:

ComparisonRoute-ranking agreementGA-pref. agreementPref. PearsonQgQ_g PearsonQjQ_j Pearson
m=1m=1 vs. m=5m=586.7%86.7%0.7480.8070.787
m=1m=1 vs. m=10m=1056.7%56.7%0.5470.6570.688

The purpose of the offline teacher is to estimate which update direction is preferable at the current training state, rather than to predict the full future optimization trajectory. This matches the online setting, where the controller is evaluated repeatedly as training evolves instead of committing to one route for many consecutive steps.

The strong agreement between m=1m=1 and m=5m=5 shows that the local preference remains stable over a moderate short horizon. At m=10m=10, the routes have undergone enough repeated updates that their parameter states increasingly diverge, so the resulting utilities reflect different optimization trajectories rather than only the preference at the original state. We therefore use m=1m=1 as a local route-preference signal that is consistent with the iterative online controller and substantially cheaper to generate.

4. Effect of bounded modulation

The modulation range [0.9, 1.1] is intentionally conservative, but its effect accumulates throughout training. On Subject 264:

MethodNovel PSNRNovel SSIMNovel LPIPSSelf PSNRSelf SSIMSelf LPIPS
Baseline30.360.9520.06629.600.9480.067
Mean Mod.31.330.9540.06130.190.9520.062
Route Mod. w/o GC31.420.9530.05830.440.9460.065
Ours31.710.9560.05730.980.9540.060

Mean Mod. uses fixed average modulation instead of adapting to each training state, while Ours performs better with state-dependent modulation. More importantly, even after completely removing group consistency, Route Mod. w/o GC still improves the Baseline by 1.06 dB on novel-view synthesis and 0.84 dB on self-reenactment. Thus, the bounded controller modulation itself has a substantial effect despite the conservative clipping range.

Thank you again for the detailed feedback. The revised manuscript and supplementary material now include the additional analyses above and clarify the corresponding implementation details.

  1. Comparison with higher-fidelity models

GaussianHeadAvatar (GHA) should have a significantly better LIPIS score than GA. At least this is what the literature typically reports, such as in NPGA. GHA also uses a CNN in screenspace. It seems that the authors are not aware of this important detail.

  1. Controller generalization beyond early training

The agreement seems suprisingly high. Is this because 10K to 120K steps does not change a lot, or bc. one route is chosen signifacntly more often than the others? What is the distribution of the chosen routes?

  1. Single-step lookahead Did you use the same batch data in all those steps? If not 86% agreement would be very surprising. Is the same batch of data required?
AuthorsOpenReview ↗

Responses to Reviewer RGdu

1. Comparison with Gaussian Head Avatar

Thank you for pointing this out. GHA does include a screen-space convolutional super-resolution network, and our reproduction used the official pipeline including this module.

For our reproduction, we followed the official GHA data-processing procedure while adopting the same NeRSemble train/test split as GaussianAvatars and MeGA for a controlled comparison. Specifically, we trained GHA from scratch for 200 epochs under this setting. During evaluation, we observed blurred facial regions and corrupted Gaussian renderings on unseen expressions, even though the final output includes the super-resolution network.

This observation is also consistent with the findings reported in the MeGA supplementary material. When reproducing GHA under the GaussianAvatars split, MeGA similarly reports degraded generalization to unseen expressions and corrupted renderings that remain visible after super-resolution.

TensorGA uses a different NeRSemble sequence split from the GaussianAvatars/MeGA protocol adopted in our evaluation. Specifically, TensorGA uses the FREE sequence as the test sequence, whereas we follow GaussianAvatars and MeGA by holding out one instructed EMO/EXP sequence and training on the remaining sequences. Therefore, its reported novel-view and self-reenactment numbers are not directly comparable with ours.

2. Controller generalization beyond early training

Thank you for raising this point. We further examined the route distribution behind the reported agreement.

At the middle stage (~65K), fresh counterfactual evidence prefers geometry for 41/50 states (82%) and appearance for 9/50 states (18%). The controller predicts geometry for 42/50 states and appearance for 8/50 states, with 90% agreement and 80.9% balanced accuracy. Thus, although geometry is already the majority route, the controller still captures part of the minority appearance-preferred states.

At the late stage (~120K), the route distribution becomes more strongly geometry-dominant: fresh evidence prefers geometry for 45/50 states (90%). In this case, the 86% agreement is partly explained by the skewed route distribution, and the balanced accuracy drops to 65.6%.

We therefore agree that the late-stage agreement should be interpreted more cautiously. The controller shows meaningful generalization at the middle stage, but as the optimization demand becomes increasingly geometry-dominant later in training, its ability to distinguish different route preferences decreases.

3. Single-step lookahead

Yes. In the multi-step analysis, all lookahead steps within a route use the same sampled mini-batch. The geometry, appearance, and joint routes also start from the same model state and use the same batch; only the trainable parameter subset differs.

In our implementation, the sampled mini-batch is fixed before entering the multi-step lookahead and reused for all m update steps. This is intentional because the purpose of the analysis is to isolate the effect of increasing the lookahead horizon from m=1 to m=5 or m=10. Using different batches across steps would mix the effect of horizon length with variation in the sampled training data.

We sincerely thank the reviewer for the careful follow-up. The questions raised here are very helpful and have prompted us to re-examine several parts of our evaluation and presentation, which has been valuable for improving the paper.

Reviewer_aiBQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper proposes counterfactual route optimization, adaptive loss weighting method for Gaussian head avatar training based on MeGA, estimating which optimization direction is more beneficial at each training state instead of fixed loss weights or handcrafted schedules. Method consists of 3 main modules: route evaluation: one-step lookahead for geometry, appearance, and joint updates; factorized route evidence; route controller, modulating loss weights during training without lookahead cost. Main contributions are counterfactual route optimization, factorized route evidence, and amortized route controller. Evaluation covers 6 baselines on NeRSemble, and route modulation ablations.

Critical strengths
  1. Paper is well-written and easy to follow, implementation is detailed, authors report offline cost, training and inference time of the method. Comparison with baselines: 6 methods. Achieving best results on novel-view and self-reenactment while inference time comparable to MeGA and Fresco.
  2. Method motivation is sound: realized outcomes of alternative updates give more direct optimization preference compared to use of fixed loss weights or handcrafted schedules
Critical weaknesses
  1. Baseline results are not fully clear. Paper follows MeGA evaluation protocol (Sec. 4.1), however MeGA achieves 30.79 PSNR on novel-view in Table 1, while original MeGA paper reports 34.11; the difference is not discussed
  2. Comparison with adaptive weighting methods is not included. Paper states that loss- and gradient-based weighting (e.g. GradNorm, uncertainty weighting) do not directly reveal the effect of updates (Sec. 1, 2.3), however no comparison is present
Questions for authors
  1. Additional ablations. Could route modulation be reported without cross-view consistency loss (Baseline + route modulation), to isolate its contribution? Could uncertainty weighting or GradNorm be added on the same setup to clarify the advantage over the prior methods?

  2. Clarification. Could you please clarify the difference between MeGA results in Table 1 (30.79 PSNR on novel-view) and the original MeGA paper (34.11) under the same protocol?

3 comments
AuthorsOpenReview ↗

Responses to Reviewer aiBQ

Thank you for the constructive comments. We have added the requested controls and clarified the evaluation setting in the revised manuscript.

1. Adaptive weighting and the contribution of route modulation

We added GradNorm and a Route Mod. w/o GC variant under the same Subject 264 setting used in our original ablation.

MethodNovel PSNRNovel SSIMNovel LPIPSSelf PSNRSelf SSIMSelf LPIPS
Baseline30.360.9520.06629.600.9480.067
Fixed-GC30.980.9540.06030.040.9500.064
GradNorm30.660.9500.05729.760.9470.064
Route Mod. w/o GC31.420.9530.05830.440.9460.065
Ours31.710.9560.05730.980.9540.060

GradNorm gives a limited improvement over the baseline, while Route Mod. w/o GC improves self-reenactment PSNR from 29.60 to 30.44 dB. The full model further reaches 30.98 dB. This shows that route modulation contributes independently of group consistency, while the consistency objective provides a complementary gain. We have added these results to the revised Table 2.

2. MeGA result in Table 1

The 34.11 dB reported in the original MeGA paper and our 30.79 dB are obtained from different evaluation sets. The original result is averaged over its nine-subject setting, whereas our Table 1 uses the common eight-subject subset and a unified evaluation protocol for all compared methods.

For completeness, we now report the full per-subject comparison between MeGA and our method.

For novel-view synthesis,

SubjectMeGAOurs
PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
07429.900.9110.10630.250.9170.105
10428.640.9180.09729.040.9250.089
25331.000.9460.05634.110.9590.033
26430.360.9520.06631.710.9560.057
30232.760.9420.04633.490.9480.042
30426.140.8600.09327.670.8710.075
30633.100.9630.03733.540.9670.035
46034.470.9620.02835.380.9670.025
Avg.30.790.9320.06631.900.9390.058

For self-reenactment,

SubjectMeGAOurs
PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
07427.090.8960.11727.390.8980.112
10427.190.9100.09927.430.9130.094
25329.870.9400.06132.660.9510.041
26429.600.9480.06730.980.9540.060
30231.530.9310.05332.180.9370.047
30427.200.8700.09227.930.8720.083
30631.930.9560.04332.240.9580.039
46032.550.9580.03833.140.9610.035
Avg.29.620.9260.07130.490.9310.063

Our method achieves higher PSNR than MeGA on all 8 subjects in both evaluation settings, while also showing consistent improvements in SSIM and LPIPS. These per-subject results have also been added to the revised supplementary material.

Thank you again for pointing out these issues.

AuthorsOpenReview ↗

Responses to Reviewer aiBQ

We further verified that MeGA’s official evaluator intersects the GT parsing mask with its own rendered raster mask before computing PSNR, SSIM, and LPIPS. As a result, regions where MeGA does not produce valid rendered coverage are masked out in both the prediction and the GT, so these missing regions do not contribute to the reconstruction error. Our evaluation does not use this model-dependent raster mask. Together with the fact that the subject set used in our evaluation is not identical to that of the original paper, these differences in the evaluation setting can account for the discrepancy between the MeGA results in Table 1 and those reported in the original paper.

AuthorsOpenReview ↗

Responses to Reviewer aiBQ

Thank you for taking the time to review our rebuttal. We also appreciate you reconsidering your rating. Your comments helped us identify points that needed clearer explanations, and we’ve revised the manuscript accordingly.

Best wishes,
The authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Area Chair, Reviewers, and Readers

Dear Area Chair, Reviewers, and Readers,

We thank all three reviewers for their careful reading and constructive feedback. Reviewers aiBQ and FC1D recognized the motivation for using the outcomes of alternative updates to guide avatar training and noted that our method maintains comparable inference time. Reviewer aiBQ also highlighted the clear presentation and the results on novel-view synthesis and self-reenactment. Reviewer FC1D appreciated the controls that isolate the contributions of individual components, and Reviewer RGdu noted the quantitative improvements.

Our revisions cover the contribution of route modulation, the comparability of baseline results, controller generalization across training stages, and the interpretation of route evidence. In particular, the new controls address how our method compares with standard adaptive weighting and what route modulation contributes on its own. We also clarified why the MeGA results in our comparison differ from the published numbers. We appreciate the reconsideration of the paper and the constructive follow-up discussion.

Our main revisions to the manuscript and supplementary material are as follows.

  1. Adaptive weighting and the contribution of route modulation. We added a comparison with GradNorm and an ablation that applies route modulation without group consistency. On Subject 264, route modulation alone improves PSNR over the baseline by 1.06 dB for novel-view synthesis and 0.84 dB for self-reenactment.

  2. Baseline evaluation and per-subject results. We explained why the MeGA results in Table 1 differ from those originally reported, clarifying the differences in subject sets and evaluation masks. We also added per-subject comparisons with MeGA and Fresco and clarified how we reproduced Gaussian Head Avatar under our evaluation protocol.

  3. Controller behavior and lookahead analysis. We evaluated the controller at the middle and late stages of training and reported balanced accuracy to account for the uneven distribution of route preferences. We also compared one-, five-, and ten-step lookahead using the same mini-batch across routes and steps to examine how route rankings change with the lookahead horizon.

  4. Method and implementation details. We clarified the respective roles of offline route evaluation and online loss-weight modulation, documented the controller’s 30 input features, and reported the per-subject offline cost separately from the avatar optimization time.

All changes are marked in blue in the revised manuscript. We hope the additional experiments and clarifications address the reviewers’ concerns and make our approach and results clearer. Thank you again for helping us improve the paper.

Best wishes,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.