Gradient Toxicity: When Auxiliary Heads Collapse the Shared Encoder
Abstract
Multi-taskoptimizationevaluateshowtaskssharegradients,buttaskcompatibility doesnotdeterminewhetherthesharedrepresentationsurvives.Weintroducegra dient toxicityasaseconddiagnosticaxis: thesignedcontributionofanauxiliary updatetorepresentation-variancecontraction. Anexactvarianceidentityshows whyconflictinformationalonecannotcertifysurvival,andtenmulti-taskoptimiz erscollapsein10/10seedsatthetestedhighdoseswhiledetachmentcollapsesin none.Becausetheaxisismeasuredbyacheapfrozenprobe, thevalidityofthat probemustbeestablishedratherthanassumed.Wederiveacomputablemargin conditionunderwhichtheprobeagreeswiththeinstantaneousvariancederiva tive,anddecomposeitsfailureintorestorationmisalignment,encodergeometry, andmismatchbetween theprobeand theoperatinggradient. Underonecon trolledtoxifier,predicteddosestrackmeasuredscattercrossingsonbatterydata butoverestimatethemby30–116×onRadioMLandby3.9–32.6×onNYUv2, andchangingonlythecalibrationtimeswitchesaHARfilterfrom10/10to0/10 collapses.Realauxiliaryheadsseparateunsignedradialloadingfromsignedcon traction,andrepresentationsurvival fromtaskutility: onNYUv2,depthRMSE improvesfrom1.33to1.09astherepresentationcontracts.Underacorrectedcell disjointbatteryprotocol,detachmentreducesstate-of-healthRMSEfrom5.57to 2.10percentagepoints. Selectivefilteringretainsausefulauxiliaryeffectwhere detachmentremovesit,butnotateverydose.Theseresultsestablishrepresenta tionsurvivalasaseparateevaluationrequirementandprobevalidityasaquantity toaudit.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.