acceptodds
Under review as a conference paper at ICLR 2027

Bounded Precision-Geometry Scaling for Robust Multi-Task Learning under Loss Scale Mismatch

Abstract

Multi-task learning often combines losses that span several orders of magnitude, causing homoscedastic uncertainty weighting to degrade severely. We propose Bounded Precision-Geometry Scaling (BPGS), a method that maps each task's log-variance through a bounded sigmoid parameterisation anchored to detached batch loss statistics, and decouples network optimisation from uncertainty optimisation. Its normalised task weights are provably invariant to uniform rescaling under non-degenerate loss scales. We evaluate BPGS on synthetic stress tests and three real-world benchmarks: NYUv2 dense prediction, Yeast multi-label classification, and RF1 multi-target regression. Under pure loss rescaling from to , its macro score changes from 0.777 to 0.778, whereas Kendall weighting drops from 0.780 to 0.637; -normalising Kendall's weights does not close the gap. On NYUv2, BPGS records the lowest depth absolute relative error (0.223), depth RMSE (0.790), and total loss (1.891) among all compared methods, including Nash-MTL. Sensitivity studies on batch size and calibration show small variation across the tested ranges, and runtime overhead relative to Kendall is under 1%. BPGS posts the highest Yeast micro-F1 (0.616) and is competitive on RF1, though PCGrad leads RMSE and MAE there. These findings establish BPGS as a scale-robust alternative to homoscedastic uncertainty weighting, notably effective when loss-scale disparities dominate multi-task optimization.

Primary area:
optimization
Keywords:
optimization, general machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DM5kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bounded Precision-Geometry Scaling (BPGS) to stabilize uncertainty-based weighting in multi-task learning with mismatched loss scales. Its main idea is to parameterize each task’s log-variance within a bounded interval determined by the current minibatch’s log-loss statistics. The resulting inverse variances are normalized into task weights. And separate stop-gradient objectives decouple network optimization from uncertainty adaptation. The authors establish batch-conditional boundedness and prove that normalized weights are invariant to uniform rescaling of all task losses under non-degenerate conditions. Experiments include NYUv2, Yeast, and RF1, showing the effectiveness of BPGS.

Critical strengths
  1. The proposed method is sound and novel.
  2. The experimental setup is comprehensive with good coverage on multiple benchmarks.
Critical weaknesses
  1. Section 4.1 of the manuscript lists the ablation variants. But it doesn't adequately define their configurations or explain how they differ from canonical BPGS. This makes it difficult to determine which components are modified in each experiment.

  2. The manuscript contains some formatting errors. This includes duplicated author names such as “Sener and Koltun Sener & Koltun (2018)” on page 2, line 82. And “Kendall et al. Kendall et al. (2018)” on page 4, line 179. Some symbols and lines overlap in Figure 1. This makes the diagram visually cluttered and low-quality. Many tables and figures in the appendix appear outside their corresponding sections, making it difficult to associate the results with the descriptions and analyses.

Questions for authors
  1. Table 6 reports scale-stress macro-score degradation of 32.3%, 28.5%, and 27.0% for Kendall, UWSO, and BPGS. Figure 5(a) reports 33.3%, 31.7%, and 29.1% for the same comparison. Could the authors explain this difference?
  2. The experimental comparisons rely on dated baselines, with Nash-MTL (2022) being the most recent. Could the authors include comparisons with recent methods published in 2025–2026?
Reviewer_wQWROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary: This paper tackles a failure mode of homoscedastic uncertainty weighting. in multi-task learning. When task losses differ by orders of magnitude, unconstrained learned log-variances turn the scale gap into large precision shifts and destabilize training. BPGS reparameterizes each task's log-variance, and ℓ₁-normalize precisions into weights.

Contributions: (1) normalized weights exactly invariant to uniform loss rescaling plus batch-conditional boundedness. (2) Scale robustness ×1–×1000 (3) NYUv2: best depth Abs Rel (0.223), RMSE (0.790), total loss (1.891), ΔM (0.078); stateless ablation collapses (mIoU 0.058, loss 6.14). (4) Stable across batch sizes 4–16 and loader seeds; <1% overhead; best Yeast metrics, competitive RF1.

Validation. Synthetic, NYUv2 (3 tasks, 120 epochs, 3 seeds), Yeast, RF1 vs Static/Kendall/UWSO/Nash-MTL/PCGrad/GradNorm-proxy; 50%-subset ablations; zᵢ/τ_T trajectories (max 0.93 init, ≤0.89 final — no boundary saturation).

Critical strengths

The rescaling experiment is clean and mechanistically tied to the proof

Critical weaknesses

Contribution confined to scale-dominated regimes by the authors' own results.

Questions for authors

N/A

Reviewer_cFpTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bounded Precision-Geometry Scaling (BPGS), a lightweight multi-task loss-weighting method that combines batch-anchored bounded log-variance parameterization, first-batch calibration, and decoupled optimization. The authors establish uniform-rescaling invariance of normalized task weights and demonstrate improved robustness under synthetic loss scaling, alongside competitive results on NYUv2, Yeast, and RF1.

Critical strengths

BPGS offers a simple, well-motivated approach with negligible computational overhead. The theoretical properties are clearly established, and the experiments demonstrate substantial robustness improvements over Kendall weighting under extreme loss rescaling. BPGS also achieves the best depth metrics and aggregate loss on NYUv2. The ablation and sensitivity studies provide useful evidence for the proposed design.

Critical weaknesses

The contribution of bounded parameterization is not fully isolated from first-batch calibration and other optimization choices. In particular, a similarly calibrated Kendall baseline is missing, while gradient-clipping thresholds differ between methods. The theoretical invariance applies to uniform rescaling, whereas the broader motivation concerns task-wise scale mismatch. Additionally, the canonical BPGS results differ substantially between Table 3 and Appendix Tables 9–10 despite apparently similar experimental settings.

Questions for authors
  1. How does BPGS compare with Kendall weighting under identical first-batch calibration and comparable optimization settings? This would help isolate the contribution of bounded parameterization.

  2. Does BPGS retain its robustness when individual task losses are independently rescaled on real-world benchmarks such as NYUv2?

  3. What explains the substantial differences in canonical BPGS performance between Table 3 and Appendix Tables 9–10? Were different training or evaluation settings used?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.