- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies incremental vision-language object detection (IVLOD) using a frozen GroundingDINO-T backbone and category-level LoRA adaptation. It identifies two failure modes: inter-category interference (where dense updates homogenize category experts) and intra-category interference (where recurring categories drift from prior learning). To address this, CADT learns and freezes a shared LoRA anchor in Task 1. DCT assigns each category an core , modulated by mean-pooled anchor descriptors, with an orthogonality penalty and amplitude alignment . For recurring categories, CAC mixes historical and current factor products via weight , re-factorizes them via SVD, and applies a SmoothL1 penalty to pull factors toward historical values. At inference, task category cores are averaged into a single update. CADT reports state-of-the-art results on ODinW-13 and ODinW-O with lower forgetting and fewer parameters.
Critical strengths
- Relevant Problem and Parameter-Efficient Category Cores: Category-level IVLOD with recurring domains is a practical setting, and storing category cores on a frozen shared anchor keeps parameter growth at per category compared to full LoRA factors.
- Comprehensive Ablation Studies: The paper provides extensive empirical evaluations and appendices covering components, parameterization, consolidation strategies, initializations, and task orders, thoroughly exploring the design space.
Critical weaknesses
W1. Multiple internal numerical inconsistencies across the text and tables: Several reported values conflict internally (e.g., Table 3's ODinW-O mismatch with Table 2's ZCOCO, identical Avg sequences in Table 13 despite differing H/ZCOCO, unmatching numbers in Table 5 and Table 4, and ambiguous parameter units for G-DINO in Table 7), undermining empirical reliability.
W2. Mismatch between theoretical motivation and actual parameterization: While theoretical sections (Appendix A.3) assume strictly orthogonal factors and direct spectral concentration, the actual DCT implementation employs unconstrained factors with only soft regularization () and exponential amplitude scaling, leaving the claimed decoupling mechanism unsupported.
W3. Unclear category-specific gradient routing mechanism: The paper never explains how individual category cores receive category-specific gradients during training; if the forward pass relies on the averaged update (Eq. 2), all cores receive identical gradients and differ only by initialization, aligning with Table 9's results.
W4. Unresolved technical foundations for the CAC fusion mechanism: The SVD re-factorization in CAC utilizes element-wise regularization that suffers from sign and rotation ambiguities, and the lack of an -only control ablation makes it impossible to determine whether performance gains stem from SVD fusion or simple regularization effects.
Questions for authors
Q1: During training, how are gradients specifically routed and distributed to each independent category core when the forward pass uses the averaged parameter update in Eq. 2?
Q2: In the CAC ablation study, what are the performance outcomes if SVD re-factorization is omitted while retaining only the constraint, and how are sign and rotation ambiguities managed within ?
Q3: In Figure 4, do boat and car correspond to recurring categories under the default task order, and what does the feature similarity matrix look like when restricting the analysis strictly to non-recurring categories?
W1. We have corrected transcription errors in our method’s ODinW-O ZCOCO in Table 3 and the full DCT configuration’s Avg and H in Table 5. Table 13 is correct. Its Avg values coincide after rounding to one decimal place, while H also depends on ZCOCO and can therefore differ. In Table 8 (originally Table 7), “M” denotes storage in MiB (220 bytes), and we have clarified this unit.
W2. Under approximate orthogonality, the squared-singular-value weighting mechanism in Appendix A.3, Eq. (42), still holds, and the approximation error can be quantitatively controlled. Ld encourages rows of M,N to be approximate orthogonality (see 6YWs W1/Q1). Eq. (42) decomposes as σ˙i=−2σi2mi⊤Γni−mi⊤Γni. The 2σi2 corresponds to the first and third terms of Eq. (38). Under approximate orthogonality, projecting these terms onto the actual singular vectors ui,vi of G gives
-u_i^\top\Gamma NS^2N^\top v_i &\approx-u_i^\top\Gamma G^\top Gv_i =-\tau_i^2u_i^\top\Gamma v_i,\\ -u_i^\top MS^2M^\top\Gamma v_i &\approx-u_i^\top GG^\top\Gamma v_i =-\tau_i^2u_i^\top\Gamma v_i. \end{aligned}$$ Adding them yields $u_i^\top\dot G_{\rm dir}v_i=-2\tau_i^2u_i^\top\Gamma v_i+r_i,$ where $\tau_i$ is the actual singular value and $r_i$ collects the approximation errors. For the first term, $NS^2N^\top-G^\top G=NS(I-M^\top M)S N^\top$, so its error is controlled by $M$'s Gram deviation. The second is similarly controlled by $N$'s. With $\epsilon=\max\{\|M^\top M-I\|_2,\|N^\top N-I\|_2\}$, summing both bounds gives \$ |r_i|\le2\epsilon(1+\epsilon)\|S\|_2^2\|\Gamma\|_2.\$ Thus, if \$ 2\epsilon(1+\epsilon)\|S\|_2^2\|\Gamma\|_2 <2\tau_i^2|u_i^\top\Gamma v_i|,\$ the error cannot reverse the predicted update sign. Eq. (42) therefore correctly predicts whether updating $M,N$ strengthens or weakens that singular component. On ODinW-13, we select multiple tasks, categories, and projections and extract the detection-loss gradient $\Gamma$ for each category core $G$. Using the same gradient, we compare Eq. (38)'s exact direction-factor update with Eq. (42)'s prediction along $G$'s leading singular direction. The tighter error bound is at most 3.45% of the predicted update magnitude, so the predicted sign is preserved. **W3/Q1.** The gradients to each category core depend on that core’s current factorization and can therefore differ across categories. Let $\bar G=\frac{1}{K}\sum_{c=1}^{K}G_c$ and $H=\nabla_{\bar G}\mathcal L_{\mathrm{task}}$. The averaging in Eq. (2) gives each core matrix the same upstream gradient, $\nabla_{G_c}\mathcal L_{\mathrm{task}}=H/K$. Backpropagating through \$G_c=M_cS_cN_c\), where $S_c=\operatorname{diag}(s_c)$, gives\begin{aligned} \nabla_{M_c}\mathcal L_{\mathrm{task}} &=\frac{1}{K}HN_c^\top S_c^\top,\ \nabla_{s_c}\mathcal L_{\mathrm{task}} &=\frac{1}{K}\operatorname{diag}(M_c^\top HN_c^\top),\ \nabla_{N_c}\mathcal L_{\mathrm{task}} &=\frac{1}{K}S_c^\top M_c^\top H. \end{aligned}
$H$ depends on the input and its supervision. The $M_c,S_c,N_c$ are fixed across samples for each category and evolve over training. Thus, changing the sample can change the factor gradients through $H$, and different category factorizations can map the same (H$ to different gradients on directions and amplitudes. Fig. 6 illustrates category specificity through gradient heatmaps computed separately for each ground-truth category in the same image. CADT’s person and horse responses focus on their respective instances, while the comparison method will activates regions of other categories. **W4/Q2.** SVD re-factorization extract new orthogonal direction factors for subsequent training. Omitting CAC’s SVD re-factorization while retaining the constraint yields 61.3 Avg, 53.2 H, and 0.2 $\bar F$ on ODinW-13, compared with 62.5 Avg, 53.7 H, and 0.1 $\bar F$ for full CADT. In this control, we skip merging and SVD re-factorization, initialize recurring-category factors from their historical values, and use frozen historical factors as regularization targets. Sign and rotation ambiguities are controlled by $\mathcal L_h$, which anchors the current factors to their frozen historical counterparts. Element-wise SmoothL1 penalizes deviations from the frozen historical factors and vanishes only when both factor pairs match. Equivalent sign flips or subspace rotations of the historical factors incur a positive penalty whenever they change the stored representation, softly anchoring subsequent learning to its signs and orientations. **Q3.** Yes, boat and car are recurring categories in Pv under the default task order. To restrict the analysis to non-recurring categories, we compute the pairwise CKA matrix of category updates on Aq using only categories absent from all preceding tasks (Fig. 8). CADT shows lower cross-category similarity, consistent with the trend among non-recurring categories in Pv (Fig. 4).