acceptodds
Under review as a conference paper at ICLR 2027

CADT: Category-aware Adaptation with Disentangled Transformations for Incremental Vision-Language Object Detection

Abstract

Incremental vision-language object detection (IVLOD) enables detectors to continuously adapt to new downstream domains while preserving pre-trained open-vocabulary generalization. However, existing methods still suffer from parameter interference during continual training. Specifically, this interference manifests in two dimensions. Regarding inter-category interference, existing category-level adaptation methods typically learn update matrices in a single parameter space, coupling the update direction and amplitude. From an optimization perspective, this coupling mechanism severely limits the degrees of freedom for category-specific adjustments. Consequently, dominant gradients from shared generic features easily overwhelm the signals of category-unique features, forcing category experts to update along generic features and resulting in severe homogenization. Regarding intra-category interference, recurring categories across tasks are updated with different task contexts, causing representation drift to accumulate task-by-task, which exacerbates forgetting. To address these issues, we propose a category-aware adaptation framework with disentangled transformations (CADT). For inter-category interference, we propose disentangled category-aware transformation (DCT). Utilizing the structure of SVD, DCT explicitly decouples the update direction and amplitude for separate optimization. This mechanism releases the degrees of freedom for parameter updates, breaks the homogenization bottleneck of optimization, and enables the independent adjustment of adaptation direction and intensity to learn category-specific features. For intra-category interference, we design category adaptation consistency (CAC), which fuses the historical and current category parameter and imposes consistency constraints to avoid excessive parameter deviation. Experiments on ODinW-13 and ODinW-O show that CADT achieves state-of-the-art performance, striking a favorable balance between category-specific adaptation and knowledge preservation.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
incremental learning, vision-language object detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5nZDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies incremental vision-language object detection (IVLOD) using a frozen GroundingDINO-T backbone and category-level LoRA adaptation. It identifies two failure modes: inter-category interference (where dense updates homogenize category experts) and intra-category interference (where recurring categories drift from prior learning). To address this, CADT learns and freezes a shared LoRA anchor (Aw,Bw)(A_w, B_w) in Task 1. DCT assigns each category an r×rr \times r core Gc=McScNcG_c = M_c S_c N_c, modulated by mean-pooled anchor descriptors, with an orthogonality penalty LdL_d and amplitude alignment LsL_s. For recurring categories, CAC mixes historical and current factor products via weight α\alpha, re-factorizes them via SVD, and applies a SmoothL1 penalty LhL_h to pull factors toward historical values. At inference, task category cores are averaged into a single update. CADT reports state-of-the-art results on ODinW-13 and ODinW-O with lower forgetting and fewer parameters.

Critical strengths
  • Relevant Problem and Parameter-Efficient Category Cores: Category-level IVLOD with recurring domains is a practical setting, and storing r×rr \times r category cores on a frozen shared anchor keeps parameter growth at O(r2)O(r^2) per category compared to full LoRA factors.
  • Comprehensive Ablation Studies: The paper provides extensive empirical evaluations and appendices covering components, parameterization, consolidation strategies, initializations, and task orders, thoroughly exploring the design space.
Critical weaknesses

W1. Multiple internal numerical inconsistencies across the text and tables: Several reported values conflict internally (e.g., Table 3's ODinW-O ΔZS\Delta_{ZS} mismatch with Table 2's ZCOCO, identical Avg sequences in Table 13 despite differing H/ZCOCO, unmatching numbers in Table 5 and Table 4, and ambiguous parameter units for G-DINO in Table 7), undermining empirical reliability.

W2. Mismatch between theoretical motivation and actual parameterization: While theoretical sections (Appendix A.3) assume strictly orthogonal factors and direct spectral concentration, the actual DCT implementation employs unconstrained factors with only soft regularization (LdL_d) and exponential amplitude scaling, leaving the claimed decoupling mechanism unsupported.

W3. Unclear category-specific gradient routing mechanism: The paper never explains how individual category cores GcG_c receive category-specific gradients during training; if the forward pass relies on the averaged update (Eq. 2), all cores receive identical gradients and differ only by initialization, aligning with Table 9's results.

W4. Unresolved technical foundations for the CAC fusion mechanism: The SVD re-factorization in CAC utilizes element-wise regularization LhL_h that suffers from sign and rotation ambiguities, and the lack of an LhL_h-only control ablation makes it impossible to determine whether performance gains stem from SVD fusion or simple regularization effects.

Questions for authors

Q1: During training, how are gradients specifically routed and distributed to each independent category core GcG_c when the forward pass uses the averaged parameter update ΔWt\Delta W_t in Eq. 2?

Q2: In the CAC ablation study, what are the performance outcomes if SVD re-factorization is omitted while retaining only the LhL_h constraint, and how are sign and rotation ambiguities managed within LhL_h?

Q3: In Figure 4, do boat and car correspond to recurring categories under the default task order, and what does the feature similarity matrix look like when restricting the analysis strictly to non-recurring categories?

1 comment

W1. We have corrected transcription errors in our method’s ODinW-O ZCOCO in Table 3 and the full DCT configuration’s Avg and H in Table 5. Table 13 is correct. Its Avg values coincide after rounding to one decimal place, while H also depends on ZCOCO and can therefore differ. In Table 8 (originally Table 7), “M” denotes storage in MiB (2202^{20} bytes), and we have clarified this unit.

W2. Under approximate orthogonality, the squared-singular-value weighting mechanism in Appendix A.3, Eq. (42), still holds, and the approximation error can be quantitatively controlled. Ld\mathcal L_d encourages rows of M,NM,N to be approximate orthogonality (see 6YWs W1/Q1). Eq. (42) decomposes as σ˙i=−2σi2mi⊤Γni−mi⊤Γni.\dot\sigma_i=-2\sigma_i^2m_i^\top\Gamma n_i-m_i^\top\Gamma n_i. The 2σi22\sigma_i^2 corresponds to the first and third terms of Eq. (38). Under approximate orthogonality, projecting these terms onto the actual singular vectors ui,viu_i,v_i of GG gives

-u_i^\top\Gamma NS^2N^\top v_i &\approx-u_i^\top\Gamma G^\top Gv_i =-\tau_i^2u_i^\top\Gamma v_i,\\ -u_i^\top MS^2M^\top\Gamma v_i &\approx-u_i^\top GG^\top\Gamma v_i =-\tau_i^2u_i^\top\Gamma v_i. \end{aligned}$$ Adding them yields $u_i^\top\dot G_{\rm dir}v_i=-2\tau_i^2u_i^\top\Gamma v_i+r_i,$ where $\tau_i$ is the actual singular value and $r_i$ collects the approximation errors. For the first term, $NS^2N^\top-G^\top G=NS(I-M^\top M)S N^\top$, so its error is controlled by $M$'s Gram deviation. The second is similarly controlled by $N$'s. With $\epsilon=\max\{\|M^\top M-I\|_2,\|N^\top N-I\|_2\}$, summing both bounds gives \$ |r_i|\le2\epsilon(1+\epsilon)\|S\|_2^2\|\Gamma\|_2.\$ Thus, if \$ 2\epsilon(1+\epsilon)\|S\|_2^2\|\Gamma\|_2 <2\tau_i^2|u_i^\top\Gamma v_i|,\$ the error cannot reverse the predicted update sign. Eq. (42) therefore correctly predicts whether updating $M,N$ strengthens or weakens that singular component. On ODinW-13, we select multiple tasks, categories, and projections and extract the detection-loss gradient $\Gamma$ for each category core $G$. Using the same gradient, we compare Eq. (38)'s exact direction-factor update with Eq. (42)'s prediction along $G$'s leading singular direction. The tighter error bound is at most 3.45% of the predicted update magnitude, so the predicted sign is preserved. **W3/Q1.** The gradients to each category core depend on that core’s current factorization and can therefore differ across categories. Let $\bar G=\frac{1}{K}\sum_{c=1}^{K}G_c$ and $H=\nabla_{\bar G}\mathcal L_{\mathrm{task}}$. The averaging in Eq. (2) gives each core matrix the same upstream gradient, $\nabla_{G_c}\mathcal L_{\mathrm{task}}=H/K$. Backpropagating through \$G_c=M_cS_cN_c\), where $S_c=\operatorname{diag}(s_c)$, gives

\begin{aligned} \nabla_{M_c}\mathcal L_{\mathrm{task}} &=\frac{1}{K}HN_c^\top S_c^\top,\ \nabla_{s_c}\mathcal L_{\mathrm{task}} &=\frac{1}{K}\operatorname{diag}(M_c^\top HN_c^\top),\ \nabla_{N_c}\mathcal L_{\mathrm{task}} &=\frac{1}{K}S_c^\top M_c^\top H. \end{aligned}

$H$ depends on the input and its supervision. The $M_c,S_c,N_c$ are fixed across samples for each category and evolve over training. Thus, changing the sample can change the factor gradients through $H$, and different category factorizations can map the same (H$ to different gradients on directions and amplitudes. Fig. 6 illustrates category specificity through gradient heatmaps computed separately for each ground-truth category in the same image. CADT’s person and horse responses focus on their respective instances, while the comparison method will activates regions of other categories. **W4/Q2.** SVD re-factorization extract new orthogonal direction factors for subsequent training. Omitting CAC’s SVD re-factorization while retaining the constraint yields 61.3 Avg, 53.2 H, and 0.2 $\bar F$ on ODinW-13, compared with 62.5 Avg, 53.7 H, and 0.1 $\bar F$ for full CADT. In this control, we skip merging and SVD re-factorization, initialize recurring-category factors from their historical values, and use frozen historical factors as regularization targets. Sign and rotation ambiguities are controlled by $\mathcal L_h$, which anchors the current factors to their frozen historical counterparts. Element-wise SmoothL1 penalizes deviations from the frozen historical factors and vanishes only when both factor pairs match. Equivalent sign flips or subspace rotations of the historical factors incur a positive penalty whenever they change the stored representation, softly anchoring subsequent learning to its signs and orientations. **Q3.** Yes, boat and car are recurring categories in Pv under the default task order. To restrict the analysis to non-recurring categories, we compute the pairwise CKA matrix of category updates on Aq using only categories absent from all preceding tasks (Fig. 8). CADT shows lower cross-category similarity, consistent with the trend among non-recurring categories in Pv (Fig. 4).
Reviewer_96NNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CADT for incremental vision-language object detection. Built on GroundingDINO, it learns category-specific direction–amplitude transformations within a shared LoRA subspace and combines SVD-based consolidation with consistency regularization for recurring categories.

Critical strengths

The paper follows a systematic academic structure, progressing clearly from the problem setting and framework overview to methodological details and optimization objectives. Its explicit distinction between new and recurring categories provides a clear organization for category-level continual adaptation.

Critical weaknesses

1. Overstated causal motivation. The introduction attributes expert homogenization to shared visual attributes producing dominant gradients that overwhelms category-specific signals. However, feature sharing alone does not imply larger gradient magnitudes or cross-category gradient alignment, leaving the proposed causal mechanism insufficiently justified.

2. Insufficient theoretical support. For example, Appendix A.3 infers near-orthogonality of the experts’ dominant singular directions from category-dependent gradients and high dimensionality without sufficient justification (Eq. 47). Consequently, the claimed reduction in expert similarity relative to direct optimization (Eq. 48) remains insufficiently supported.

3. Marginal and inconsistent empirical gains. CADT achieves only marginal improvements over competing methods, with substantial performance deficits on more than half of the evaluated subtasks. These results provide limited evidence of a meaningful and consistent practical advantage over existing approaches.

4. Limited technical novelty. Core components have direct precedents in existing methods (e.g., LoRA-XS and AdaLoRA), yet are presented as key contributions of this work. Reusing established techniques is not inherently problematic; the concern is whether the remaining modifications constitute a substantive advance. Given the limited theoretical support and inconsistent empirical gains, the paper provides insufficient evidence of a substantial technical contribution.

Questions for authors

Please refer to weakness section.

1 comment

W1. Shared visual attributes may contribute to homogenization when their gradient contributions are sufficiently strong and aligned across categories. CADT’s factorization allows the shared upstream gradient to produce different direction and amplitude updates across category cores (see 5nZD W3/Q1). In Fig. 6, DitHub and LoRA-XS show overlapping responses around the person–horse boundary and cow–person interface. CADT’s person and horse responses focus on their respective instances, while the comparison methods also activate regions of other categories. These observations complement the lower cross-category similarity measured in Fig. 4.

W2. To quantify how CADT affects expert similarity, we compare it with Direct-GG, which optimizes each category core GcG_c directly. We compute the SVD of each learned core and compare category pairs within the same layer[1]. Top-1 overlap is the absolute cosine between the leading left or right singular vectors. Rank-1 cosine is the Frobenius cosine between the leading rank-1 approximations. Top-4 overlap is ∥Qi⊤Qj∥F2/4\|Q_i^\top Q_j\|_F^2/4, where QcQ_c contains the top four left or right singular vectors. Top-4 diversity is one minus the mean of the left and right overlaps. Core cosine is the Frobenius cosine between the full cores, as in Eq. (48). Higher overlaps and cosines indicate stronger alignment of directions, subspaces, or updates. Higher diversity indicates less subspace overlap. For each metric, we average over matched layer–category pairs within each task and then weight the seven multi-category ODinW-13 tasks equally.

MetricCADTDirect-GGΔ\Delta
Left Top-1 overlap ↓0.77410.8829−0.1088
Right Top-1 overlap ↓0.77520.9001−0.1249
Rank-1 cosine ↓0.70860.8489−0.1403
Left Top-4 overlap ↓0.77350.8783−0.1048
Right Top-4 overlap ↓0.77480.8930−0.1182
Top-4 diversity ↑0.22590.1143+0.1116
Core cosine ↓0.84410.9164−0.0723

Δ\Delta is CADT minus Direct-GG before rounding. With the soft Ld\mathcal L_d penalty (see 6YWs W1/Q1), CADT shows lower cross-category overlaps and core cosine and higher Top-4 diversity, supporting reduced expert similarity.

[1] Hamm and Lee. Grassmann Discriminant Analysis. ICML, 2008.

W3. CADT achieves state-of-the-art Avg and H on both ODinW-13 and ODinW-O in our full-data comparisons (Tables 1 and 2). Across four seeds (see 6YWs W2/Q3), CADT improves mean Avg over DitHub by 0.60 and 0.67 AP points, respectively, with corresponding H gains of 0.27 and 0.32 points. CADT improves both metrics under all three tested ODinW-13 task orders (Table 9). With three category groups, the gains reach +1.6 Avg, +0.9 H, and +0.2 ZCOCO (Table 15). On ODinW-13, CADT reduces adaptation parameter storage from 147.0 M for DitHub to 21.6 M (Table 8), an 85.3% reduction. We also evaluated CADT on AODRaw (Li et al., CVPR 2025), which covers nine domains with substantial category overlap. Compared with DitHub, CADT achieves higher AP in six of the nine domains while using fewer adaptation parameters.

MethodZCOCOAvgHFog and Low LightFog and RainFogIndoor Low LightIndoor Normal LightLow Light and RainLow LightNormal LightRain
GD ZS47.415.623.613.319.619.415.824.811.716.35.814.1
ZiRA44.432.337.427.636.329.328.343.230.731.730.533.5
DitHub45.234.038.830.335.531.431.047.831.432.930.935.1
CADT45.534.639.331.035.532.830.349.132.533.030.536.4

W4. Our method introduces a novel design that differs substantially from existing methods and delivers better performance. AdaLoRA determines how much rank each layer retains. LoRA-XS learns a single dense core to reduce the number of trainable parameters. Extending it to one core per category (Direct-GG) yields highly similar dominant components across categories in our experiments. CADT’s main contribution is an SVD-inspired factorization that yields more distinct updates across categories. CADT also achieves higher downstream AP than the evaluated direction–magnitude and weight decomposition baselines (see 6YWs W2/Q2). For a fair comparison, we compare AdaLoRA and LoRA-XS in category-level adaptation without auxiliary loss constraints. The results on ODinW-13 are shown below.

MethodZCOCOAvgHAeAqCoEgMuPaPvPiPoRaShThVe
AdaLoRA-style47.055.751.028.125.868.772.837.959.860.370.154.474.134.676.461.4
LoRA-XS-style46.958.352.034.240.171.670.148.453.168.468.847.272.539.378.165.9
CADT47.360.553.136.843.674.472.050.558.369.769.550.873.841.977.567.5

CADT achieves the highest ZCOCO (47.3), Avg (60.5), and H (53.1), with Avg gains of 2.2 and 4.8 AP points over the LoRA-XS-style and AdaLoRA-style baselines, respectively.

Reviewer_6YWsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates parameter-efficient continual adaptation in incremental vision-language object detection. The authors identify two types of interference that arise when models continually learn new downstream domains: inter-class interference, where the adaptation parameters for different classes become coupled in terms of direction and magnitude within a shared low-rank space—leading to convergent updates and a lack of class-specific distinctiveness; and intra-class interference, where a single class is repeatedly updated using different contexts across various tasks, resulting in representation drift and forgetting. To address these issues, the authors propose two complementary components. First, inspired by singular value decomposition (SVD), they explicitly decouple the low-rank adaptation matrix for each class into a directional component and a magnitude-modulation component; this allows for independent optimization of direction and magnitude, thereby overcoming the bottleneck of convergent updates across classes. Second, for recurring classes, they integrate previously learned directional information with the current direction and impose a consistency constraint, which mitigates representation drift for the same class across different tasks. The proposed method is evaluated on the ODinW-13 benchmark and a newly introduced benchmark with high class overlap, ODinW-O. Compared against methods such as G-DINO, iDETR, OWL-DETR, ZiRa, DiHub, and DGS—using metrics including zero-shot retention, average performance on learned tasks, harmonic mean, and forgetting rate—the approach achieves state-of-the-art performance on both benchmarks. Furthermore, the contributions of the individual components are validated through 12 ablation studies.

Critical strengths
  1. The problem decomposition is clear, with a well-aligned component design and thorough ablation studies. The authors break down the interference encountered during continuous adaptation into two orthogonal dimensions—inter-class and intra-class—addressing them respectively through decoupled transformations and consistency constraints. The step-by-step ablation analysis in Table 4 demonstrates that each added component yields quantifiable gains in both the accuracy on previously learned tasks and the harmonic mean, providing a clear attribution of the benefits to each component.
  2. The method offers significant advantages in terms of parameter efficiency. As shown in Table 7, with a rank of 16, the proposed approach utilizes only ~21.6 million parameters—approximately 85% fewer than the 147 million parameters required by the baseline method, DiHub—while simultaneously achieving higher accuracy on previously learned tasks and a better harmonic mean. Furthermore, Figure 6 illustrates that the growth of the parameter count relative to the number of classes is notably gradual, which is highly advantageous for practical deployment.
Critical weaknesses
  1. There is a disconnect between the core theoretical analysis and the actual implementation. The derivation in the appendix regarding the optimization dynamics of Singular Value Decomposition (SVD) relies on the assumption that directional components remain strictly orthogonal; based on this, it is argued that "SVD-based optimization concentrates adaptation energy into the dominant directions." However, in the actual implementation, directional components are constrained only via a "soft" orthogonality regularization term rather than a "hard" constraint, meaning they inevitably deviate from orthogonality during training. Consequently, the key premise from the appendix—that the diagonal elements of an antisymmetric matrix are zero—does not strictly hold in practice. While the introduction and the list of contributions highlight the "theoretical proof that dense optimization leads to convergent category updates" as a primary selling point, this explanation remains largely idealized, lacking empirical monitoring of the extent to which directional components deviate from orthogonality during the training process.
  2. The paper lacks comparisons with the most relevant decomposition-based methods, and the reported performance gains have not been subjected to significance testing. The authors’ approach—decomposing low-rank adaptation weights into directional and magnitude components—is highly similar to the DoRA (Weight-Decomposed Low-Rank Adaptation) method proposed in 2024. Yet, DoRA is neither mentioned in the related work section nor included in the main experimental tables, nor is there a comparison under an equivalent parameter budget. Furthermore, in the main results, the proposed method shows an improvement of only about 1–2 percentage points in accuracy on previously learned tasks compared to DiHub. The paper fails to report the mean and standard deviation across multiple random seeds, and the experiments on robustness to task ordering provide only mean values without indicating the range of variation; thus, it is difficult to determine whether these performance gains are stable.
Questions for authors
  1. The derivation in the appendix assumes that the directional components are strictly orthogonal; however, in practice, this is constrained only by a soft regularization term. Could the authors report—in the rebuttal—how the deviation of these directional components from an orthonormal matrix evolves over the course of training? If the deviation is significant, does the claim in the appendix—that "SVD-based optimization concentrates adaptation energy into the dominant directions"—still hold?
  2. What is the fundamental difference between the authors' direction-magnitude decomposition and the weight decomposition used in DoRA? Why not simply adopt DoRA as the parameterization for class-specific adaptation? Could the authors provide a comparative experiment using DoRA for class-specific adaptation, keeping the total parameter budget consistent?
  3. Could the authors report the mean and standard deviation (across at least three random seeds) for the results in the main table and the table concerning robustness to task ordering? Given that the observed improvements are only 1–2 percentage points, it is difficult to determine their statistical significance without error bars.
  4. During inference, the adaptation updates for all classes within the current task are averaged and added to a shared low-rank anchor. Does this process negate the class-specificity that the method aims to achieve? Would it be more appropriate to apply varying update intensities for different classes within the same task, rather than averaging them uniformly?
  5. While the authors emphasize a substantial reduction in the number of parameters, the paper does not report computational overhead (e.g., FLOPs or per-step latency) for training or inference. Does a lower parameter count necessarily translate to faster inference? In the context of edge deployment, what is the actual advantage of this method over DiHub in terms of throughput or latency?
1 comment

W1/Q1. We acknowledge that Ld\mathcal{L}_d imposes a relatively weak orthogonality constraint, but our experiments show that even without exact orthogonality, making the directional factors more orthogonal can concentrate the adaptation more strongly along its dominant directions. Specifically, we quantify their deviation from orthonormality using δ=∥M⊤M−I∥F+∥NN⊤−I∥F2r,[1]\delta=\frac{\|M^\top M-I\|_F+\|NN^\top-I\|_F}{2\sqrt r}, [1]

where δ=0\delta=0 indicates exact orthonormality. The final deviation is 0.2276 with Ld\mathcal L_d and 1.2742 without it. We additionally measure spectral concentration using E4(G)=∑i=14σi2∑i=116σi2,[2]E_4(G)=\frac{\sum_{i=1}^{4}\sigma_i^2}{\sum_{i=1}^{16}\sigma_i^2}, [2] where σi\sigma_i are GG’s singular values in descending order. The four leading singular directions capture 85.13% of the total spectral energy with Ld\mathcal L_d, compared with 54.87% without it, supporting energy concentration under the soft constraint.

Our analysis accommodates approximate orthogonality with an explicit error bound (see 5nZD W2).

[1] Bansal et al. Can We Gain More from Orthogonality Regularizations in Training Deep Networks? NeurIPS, 2018.

[2] Li et al. X-EcoMLA: Upcycling Pre-Trained Attention into MLA for Efficient and Extreme KV Compression. COLM, 2025.

W2/Q2. DoRA scales entire output channels of the adapted weight, and CADT scales rank-one components of a category-specific additive update. In DoRA, WDoRA′=S(W0+BA)W'_{\mathrm{DoRA}}=S(W_0+BA), so each diagonal entry of SS scales both the pretrained weights and the low-rank correction in the corresponding output channel. In CADT, Wc′=W0+BwMcDcNcAwW'_c=W_0+B_wM_cD_cN_cA_w. Each diagonal entry of DcD_c weights the outer product of a column of BwMcB_wM_c and a row of NcAwN_cA_w. A single CADT amplitude can therefore affect multiple output channels, and the full category-specific update has rank at most rr.

We evaluated category-wise DoRA in the DitHub pipeline. Each category stores its own rank-16 Ac,BcA_c,B_c and magnitude vector. The results are shown below. ODinW-13 — CADT vs. DoRA

MethodParamsZCOCOAvgHAeAqCoEgMuPaPvPiPoRaShThVe
CADT21.6M47.062.553.737.645.874.072.354.562.770.271.051.977.145.681.268.5
DoRA317.6M48.158.052.636.531.369.474.142.257.763.771.657.376.535.374.364.2

ODinW-O — CADT vs. DoRA

MethodParamsZCOCOAvgHAeHhPvScThVe
CADT8.6M47.059.952.750.568.072.842.372.852.9
DoRA36.1M47.058.352.049.067.068.841.175.148.5

Compared with category-wise DoRA, CADT improves Avg by 4.5 and 1.6 AP points and H by 1.1 and 0.7 points on ODinW-13 and ODinW-O, respectively, while reducing adaptation parameters by 93.2% and 76.2%.

W2/Q3. We shuffled the task order separately for each seed (0, 42, 1024, and 2027) and used the same order for both methods within each seed. The table reports the mean and standard deviation across the four runs in the full-shot setting.

BenchmarkMethodZCOCOAvgH
ODinW-13DitHub47.25 ± 0.4561.75 ± 0.5553.54 ± 0.24
ODinW-13CADT47.33 ± 0.1562.35 ± 0.4053.81 ± 0.25
ODinW-ODitHub47.03 ± 0.1759.43 ± 0.7452.51 ± 0.19
ODinW-OCADT47.13 ± 0.2160.10 ± 0.4352.83 ± 0.08

CADT gains 0.60/0.67 Avg and 0.27/0.32 H points on ODinW-13/ODinW-O. On AODRaw (Li et al., CVPR 2025), CADT achieves 45.5/34.7/39.4 ZCOCO/Avg/H vs. DitHub’s 45.2/34.0/38.8 (see 96NN W3).

Q4. Each category contributes its own learned update to the shared task adapter under uniform averaging. Backpropagating the shared upstream gradient through each category’s factorization can yield different factor gradients, allowing each category to learn a distinct update under the same task loss (5nZD W3/Q1). To test the proposed weighting, we rescaled each category update by a learned scalar γc\gamma_c before averaging.

Category UpdateODinW-13 ZCOCOODinW-13 AvgODinW-13 HODinW-O ZCOCOODinW-O AvgODinW-O H
Gc=McScNcG_c = M_c S_c N_c (ours)47.062.553.747.059.952.7
Gc=γcMcScNcG_c = \gamma_c M_c S_c N_c46.761.353.046.758.952.1

Compared with uniform averaging, learned weighting lowered Avg by 1.2 and 1.0 AP points and H by 0.7 and 0.6 points on ODinW-13 and ODinW-O, respectively.

Q5. We compare the computational overhead of DitHub and CADT during inference on ODinW-13.

Inference metricDitHubCADT
Latency (ms/image)123.689.9
Throughput (images/s)7.98.9
Peak reserved GPU memory (GiB)2.51.7

Relative to DitHub, CADT's estimated per-image latency is 27.3% lower, and its estimated peak reserved GPU memory is 32.0% lower. Both methods precompose task-level update. For this step, CADT averages r×rr\times r category transformations, and DitHub averages r×dinr\times d_{\mathrm{in}} category-specific LoRA component.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.