- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies which task-support patterns identify block-specific latent representations and whether acquiring more tasks helps under a fixed labeling budget. It characterizes the remaining mixing ambiguities and introduces a fractional-cover bound that combines task errors, conditioning, and support overlap. The paper also proposes linear and nonlinear estimators. Experiments show acquisition benefits in a controlled Gaussian setting, while nonlinear and image results are less conclusive.
Critical strengths
-
The fractional-cover bound goes beyond a binary identifiability result. It combines overlapping task constraints without repeatedly charging shared nuisance directions. LP duality establishes optimality within the local-variance relaxation, and a constructed example shows a concrete improvement over the coordinatewise bound. This helps explain how support geometry and conditioning affect recovery after identifiability is established.
-
The experimental comparisons are carefully controlled. The authors match labeling budgets and support information, include a baseline that nearly matches PATI, and report inconclusive and negative results. The nonlinear follow-up and image benchmark make the limits of the current method visible rather than presenting only favorable outcomes.
Critical weaknesses
-
The experiments do not clearly show the added benefit of fractional-cover aggregation. The identifiability results build on prior work, so the significance of the paper depends heavily on this quantitative refinement. Yet its advantage is mainly illustrated by constructed examples, without a direct empirical comparison with the coordinatewise bound in Theorem 3. Figure 1(d) also reports a median bound-to-observed-variance ratio of 6.31, suggesting considerable slack. The acquisition experiments mostly show the value of separating tasks and choosing better-conditioned candidates. Beyond the controlled linear setting, the nonlinear gain has a confidence interval containing zero, and the image experiments do not establish consistent improvements.
-
It is unclear how the bound could be used with real tasks. Theorem 4 requires population signature errors and inverse-Lipschitz constants, which are not directly available from labels. The synthetic experiments use access to ground truth to estimate these quantities. Although the paper acknowledges this limitation, it leaves unclear how a practitioner could compute a reliable bound or use it to guide task acquisition.
Questions for authors
-
Can you compare the fractional-cover and coordinatewise bounds on trained representations across different overlap patterns, while controlling for identifiability and task conditioning? This would help isolate the contribution of fractional-cover aggregation from the benefits of support separation alone.
-
Under what assumptions can the signature errors and inverse-Lipschitz constants be estimated or upper-bounded from pilot or validation data? Even a conservative data-dependent bound with a stated confidence level would help. If additional assumptions are essential, please explain what information is missing from labels alone.
-
Do the acquisition gains persist when candidate tasks have similar conditioning or fewer pilot samples are available? In Section 6.1, the separating candidates have substantially different strengths ( and ), and the policy always selects the stronger one. Tests with other support patterns and budget-to-dimension ratios, under matched total budgets, would clarify how broadly this result applies.
-
What causes the shifted-accuracy decline in the six deteriorating nonlinear follow-up seeds: optimization failure, poor inverse conditioning, or remaining within-block ambiguity? Diagnostics for individual seeds would help explain the limits of PATI-Flow.