acceptodds
Under review as a conference paper at ICLR 2027

Learning What to Preserve: Target-Directed Information Bottleneck for Infrared Small Target Detection Under Degradation

Abstract

Information preservation is essential for infrared small target detection, but indiscriminate preservation becomes a liability when degradation dominates the input. Conventional encoders make retention an implicit consequence of task optimization, whereas invertible encoders make information recoverable without deciding what is worth retaining. This work introduces Target-Directed Information Bottleneck (TDIB), which unifies selective information retention with inference-aligned detection. Target-Directed Selective Retention (TDSR) integrates a frozen invertible prefix and a variational bottleneck, confining information loss to the bottleneck and making its cost measurable through the fixed inverse. Its spatially weighted KL rate drives information compression, while localized inverse reconstruction protects fragile target evidence without requiring clean references. Inference-Aligned Ranking (IAR) further turns posterior variation into supervision for reliable detection, encouraging mean–sample prediction consistency and ranking candidates by localization quality discounted by posterior drift. Comparisons and ablations on the proposed IR-DegradeX benchmark show that TDIB consistently improves detection accuracy, while representation analyses confirm that it preserves target evidence and stabilizes predictions. These findings point to a principled pathway toward target-directed representation learning for degraded visual perception.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Variational Information Bottleneck, Invertible Neural Networks, Infrared Small Target Detection, Selective Information Retention, Degradation-Robust Detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_W3hBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Target-Driven Information Bottleneck (TDIB) framework for infrared small target detection under degradation. The method introduces Target-Driven Selective Retention (TDSR), which combines a frozen invertible encoder and a variational bottleneck to preserve target information through spatially weighted KL divergence and local inverse reconstruction. Inference-Aligned Ranking (IAR) uses posterior sampling to improve prediction consistency and candidate box ranking. The authors also construct IR-DegradeX with multiple degradation settings. Experiments show improvements in overall AP, particularly at stricter IoU thresholds.

Critical strengths
  1. The idea of selective information retention is interesting. It provides a different perspective from conventional image restoration and feature purification methods for degraded infrared small target detection.
  2. The method design is reasonable. TDSR focuses on preserving target information, while IAR uses posterior sampling to improve prediction consistency and candidate ranking.
  3. The experiments include multiple degradation settings, multi-seed ablations, and representation analysis. The results show improvements in overall AP, particularly at stricter IoU thresholds.
Critical weaknesses
  1. TDSR reduces information compression in target regions but does not explicitly distinguish ordinary backgrounds from target-like clutter. Whether this design can effectively suppress false alarms needs further verification. Besides, some background context may be useful for distinguishing real targets from clutters. Strong background compression may remove such information.
  2. IAR combines posterior drift and localization quality for candidate ranking. However, the posterior drift values in Figure 4 are relatively small, making r_i=q_i\exp(-d_i) very close to q_i. Whether posterior drift provides additional benefits beyond localization quality is unclear. Besides, the ablation results show that consistency supervision reduces prediction disagreement and localization jitter but slightly decreases AP. The relationship between prediction stability and detection performance needs further explanation.
  3. The experiments are mainly conducted on IR-DegradeX, where the training and test sets use the same synthetic degradation operators. Generalization to unseen degradations and real-world interference remains unclear. Besides, the results mainly report overall AP under different degradation combinations. More detailed evaluations of individual degradation types, Precision, Recall are needed to clarify the sources of performance improvements.
Questions for authors

Refer to weakness.

6 comments
AuthorsOpenReview ↗

Reply 1/2: Background clutter, posterior drift, and IAR

W1. Target-like clutter, background context, and false alarms. TDSR allocates compression pressure according to annotated target regions rather than intensity or contrast. Bright target-like clutter therefore receives no automatic target-level protection, while background structure can remain available when useful for discrimination. Detection supervision determines which retained responses become target predictions.

We tested whether useful context survives selection using 740 target/background patch pairs matched in size, intensity, and local contrast. Full TDIB reconstructs target and matched-background patches with average errors of 0.078 and 0.136 gray levels, versus 3.666 and 4.524 without local fidelity (A2). Figure 9 shows that retained information need not elicit a high detection response: visible date characters retain nearly the same raw posterior KL as the UAV (20.82 vs. 21.86) but receive a peak detection score of 0.0044 vs. 0.8237 for the target. On the test FROC curves at matched 95% Pd (855/900 targets), Full produces 16 false boxes (0.0178/image), versus 11 (0.0122/image) for A1 (uniform rate), while improving Macro AP from 67.92 to 68.19. These results show that TDSR preserves background context without requiring it to produce target-like detector responses. (Sec. 4.3; Appendix 12.3, Tables 28–29 and Fig. 9)

W2. Posterior drift and candidate ranking. To isolate drift, we replaced the reliability target r=qexp⁡(−d)r=q\exp(-d) with localization quality qq alone, keeping assignment and all other losses unchanged:

Ranking targetSeed 0Seed 1Seed 2Mean ± SD
Quality only, qq67.9868.2367.1967.80 ± 0.54
Drift-discounted, qexp⁡(−d)q\exp(-d) (Full)68.1968.2868.1568.20 ± 0.07

Drift discounting improves AP in all three seeds (+0.40 AP on average) and reduces the standard deviation from 0.54 to 0.07. It therefore adds useful supervision beyond localization quality alone. Fig. 4 reports the dimensionless drift of Eq. (5), computed from the original predictions with center displacement normalized by the target diagonal. Drift stays below 0.007 for all three illustrated candidates, so rr remains close to qq. Both qq and rr favor c4c_4 over c3c_3, whereas the mean-path score ranks c3c_3 first; the ranking loss addresses this disagreement. The figure illustrates how reliability supervision is constructed in one example, while the three-seed control above measures the contribution of drift discounting to detection. Reliability is a training target, not test-time reranking.

Consistency controls a different property. Across three seeds, adding consistency to A3 decreases classification JS from 35.39 to 11.03 (×10⁻⁶) and box jitter from 0.0256 to 0.0183 px, while AP stays nearly unchanged (67.89 → 67.82). Stable predictions are not necessarily correctly ranked. The controlled 2×2 ablation makes their complementarity explicit:

No rankingRanking
No consistency67.913 (A3)67.491
Consistency67.844 (A4)68.187 (Full)

With consistency, ranking contributes +0.343 AP; without it, ranking changes AP by −0.422. Consistency makes predictions robust to posterior variation, while drift-discounted ranking teaches which candidate should receive the higher deployed score. Their combination turns posterior variation into more reliable single-path detection, which is the purpose of IAR. (Sec. 4.3; Appendix 11.2, Tables 23–24; Appendices 12.4 and 12.6)

AuthorsOpenReview ↗

Reply 2/2: Individual degradations and real-image generalization

W3. Operator-level AP, precision/recall, and generalization. We added results for the seven single-degradation groups in the 300-image IR-DegradeX L1 test split. Entries are COCO box AP and precision/recall at IoU 0.50 with validation-selected thresholds. “Best baseline” is the highest per-operator AP among JFD³, NS-FPN, and UIU-Net; the strongest baseline varies by operator.

Operator (images)TDIB APBest baseline APTDIB precision / recall (%)
Gaussian blur (49)68.2666.3197.92 / 95.92
Motion blur (64)69.5268.6898.36 / 93.75
Infrared haze (54)71.2871.08100 / 100
Gain nonuniformity (49)64.6164.76100 / 97.96
Gaussian noise (28)68.5468.79100 / 100
Fixed-pattern stripes (30)67.4668.65100 / 100
Bias field (26)67.8769.05100 / 96.15

Across the seven operators, TDIB achieves the highest average AP (68.22) and the best worst-operator AP (64.61), compared with 67.55/63.44 for JFD³, 66.17/61.09 for NS-FPN, and 65.28/60.91 for UIU-Net. TDIB leads on Gaussian blur, motion blur, and infrared haze and remains close to the per-operator best elsewhere. Its drop from the operator mean to the worst operator is also the smallest (3.61 AP, versus 4.11–5.08 for the baselines). TDIB therefore shows the most uniform performance across degradation types, rather than depending on one favorable operator. The revised appendix reports AP and precision/recall for all four methods on each operator. (Appendix 9.2, Tables 16–17)

We further evaluated independent real-image generalization on IRBlurUAV. All methods were adapted and selected using only synthetic IRBlurUAV data; the 4,118 real frames were held out for testing.

MethodReal test AP / AP50 / AP75 (4,118 frames)
TDIB15.00 / 48.99 / 3.85
JFD³13.13 / 44.06 / 2.10
NS-FPN12.41 / 41.06 / 3.04
UIU-Net9.03 / 35.87 / 1.18

These frames are captured by infrared sensors rather than generated by synthetic operators, so their blur includes sensor and scene effects that simulation can only approximate; none was used for adaptation or model selection. TDIB leads in AP, AP50, and AP75 on real images, improving AP by 1.87 points over the strongest alternative. Together with the seven-operator results, this provides complementary evidence across constructed degradation processes and independently acquired infrared imagery. (Sec. 4.2, Table 2; Appendices 9.2 and 10.1)

After reading the repsonse, I still concerned about the generalization capability of this method. Although the authors subsequently expanded their evaluation to include results on IRBlurUAV, these datasets are not commonly used in this field. I suggest that the authors validate the performance on IRSTD-1K and NUAA-SARST datasets(for degraded scenarios, you can introduce noise or other forms of interference into these datasets). Consequently, I am maintaining my previous score.

Reviewer_r3xfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Target-Directed Information Bottleneck (TDIB) for infrared small target detection under degraded imaging conditions. The main idea is to retain information useful for detecting small targets rather than preserving everything in the input, including degradation artifacts. The framework contains two components. Target-Directed Selective Retention (TDSR) combines a frozen invertible encoder with a variational bottleneck. It relaxes the compression penalty near target regions and uses local reconstruction through the fixed inverse to protect target details. Inference-Aligned Ranking (IAR) then uses posterior samples to improve the consistency and ranking of predictions made by the deterministic inference path. The method is evaluated on IR-DegradeX, a benchmark constructed with single and compound degradations. The paper reports comparisons with existing detectors, ablation results across three random seeds, and analyses of the learned representations.

Critical strengths

1.The motivation is clear. For infrared small targets, preserving all input information may also preserve noise and other unwanted artifacts. The combination of a variational bottleneck and a fixed invertible encoder offers a reasonable way to control information loss while protecting target regions. I also find the separation between spatial rate allocation and local reconstruction useful, since these two constraints address different aspects of target preservation. 2.The experimental analysis goes beyond reporting detection accuracy alone. Table 2 includes three-seed ablations and measures reconstruction error, posterior variation, and prediction stability. The IoU-threshold analysis in Figure 6 is also useful, as it shows where the improvement in overall AP comes from rather than simply reporting a single aggregate score.

Critical weaknesses

1.The missing comparison with InvDet is my main concern. Section 3.1 states that the invertible encoder follows InvDet, but this method is absent from Table 1. Although the A0 ablation removes the bottleneck, it does not reproduce the original InvDet framework and its reconstruction-guided training. A direct comparison is therefore important for understanding the contribution of TDIB. In addition, the improvements over JFD3 and NS-FPN are about 1.34 and 1.46 AP points, respectively, while the main baseline results are based on single runs. Without repeated evaluations of these competitors, it is difficult to judge how reliable the reported margins are. 2.I am also concerned about how well the results transfer beyond IR-DegradeX. The benchmark uses synthetic degradations applied to a fixed set of source images, which is useful for controlled evaluation but does not necessarily reflect real sensor noise or more complicated imaging conditions. No independent real-world degradation benchmark is included in the reported comparisons. Another point is that TDIB has the highest overall AP but falls behind some baselines on AP50 and AP75. The gains at stricter IoU thresholds are interesting, but their practical value would be clearer with additional detection-oriented metrics and tests on real degraded infrared images.

Questions for authors

1.Could the authors compare TDIB directly with InvDet using the same data and training protocol? It would also help to report multi-seed results for JFD3 and NS-FPN. 2.How does TDIB perform on an independent infrared dataset with real sensor degradations or corruption patterns not included in IR-DegradeX? 3.Since the improvement is concentrated at higher IoU thresholds, could the authors also report detection probability, false-alarm rate, or localization error to clarify its practical impact?

1 comment
AuthorsOpenReview ↗

InvDet, repeated baselines, real degradation, and practical localization

W1/Q1. InvDet and repeated baselines. We trained InvDet and TDIB on the same degraded IR-DegradeX LMix images and box annotations, with validation-based checkpoint selection. InvDet retains its native target assignment and reconstruction objective.

MethodMacro APAP50AP75
InvDet42.8592.5831.44
TDIB68.1998.0376.02

At the method level, TDIB exceeds InvDet by 25.34 Macro AP, and the gap widens from 5.45 AP50 to 44.58 AP75 as localization becomes stricter. Both methods build on invertible representations but differ in their complete designs: whether information is selected before detection, how detection features are built from the invertible representation, and their training objectives. The complete-method comparison demonstrates the advantage of TDIB's overall design for precise localization using the same degraded images and box annotations.

The shared-foundation Full/A0 control further isolates the contribution of the proposed retention framework within one detector. A0 retains TDIB's invertible foundation and detector but removes the bottleneck and its associated objectives, with the remaining training settings matched. Across three matched seeds, Full improves Macro AP from 67.32 to 68.20 (+0.88) and AP85 from 43.38 to 46.68 (+3.30), with a positive AP85 gain in each run. The stronger gain at strict overlap shows that learning what to preserve improves precise localization even when a recoverable representation is already available.

We also repeated the two strongest degradation-aware baselines:

MethodSeed 0Seed 1Seed 2Mean ± SD
JFD³66.8466.4366.2766.51 ± 0.29
NS-FPN66.7366.5765.9466.41 ± 0.42
TDIB68.1968.2868.1568.20 ± 0.07

Every TDIB run exceeds every baseline run: even its lowest (68.15) exceeds the strongest baseline run (66.84). The three-seed mean margins are 1.69 AP over JFD³ and 1.79 AP over NS-FPN, with the smallest spread. (Sec. 4.2, Table 1; Sec. 4.3, Table 3 and Fig. 6(b); Appendices 8.3 and 9.1, Tables 13–14)

W2/Q2. Independent real-degradation benchmark. We added IRBlurUAV, initializing from IR-DegradeX checkpoints. Adaptation and checkpoint selection used only synthetic IRBlurUAV data; the 4,118 real frames were used only for testing. JFD³ uses degraded-only supervision, without paired-clean losses.

MethodSynthetic AP / AP50 / AP75 (2,880 images)Real AP / AP50 / AP75 (4,118 frames)
TDIB31.52 / 67.38 / 23.4115.00 / 48.99 / 3.85
JFD³23.49 / 57.98 / 12.9413.13 / 44.06 / 2.10
NS-FPN25.93 / 60.28 / 14.9912.41 / 41.06 / 3.04
UIU-Net19.26 / 54.63 / 6.959.03 / 35.87 / 1.18

TDIB leads on every metric, including AP50 and AP75, where it trailed the strongest baselines on IR-DegradeX. The real test frames are captured by infrared sensors rather than generated by synthetic operators, so their blur includes sensor and scene effects that simulation can only approximate; none enters adaptation or checkpoint selection. TDIB improves AP by 1.87 points on these real frames and by 5.59 points on the synthetic test set over the strongest alternatives, supporting transfer beyond the constructed degradation benchmark. (Sec. 4.2, Table 2; Appendix 10.1)

W2/Q3. Detection probability, false alarms, and localization error. We added Pd, false boxes per image, and IoU-based localization with validation-selected thresholds. Center Pd uses one-to-one matching within 3 native pixels.

MethodCenter Pd (%)False boxes/imagePrecision / recall @ IoU 0.85 (%)Correct @ IoU 0.85
TDIB96.110.018956.69 / 55.56500/900
JFD³97.220.006752.10 / 51.00459/900
NS-FPN95.560.025646.66 / 45.78412/900
UIU-Net95.670.017853.36 / 52.00468/900

All four methods have Pd of 95.56–97.22% and fewer than 0.03 false boxes/image; their IoU 0.50 recall differs by at most one point. Tight localization distinguishes them: TDIB outperforms JFD³ and NS-FPN at every evaluated IoU threshold from 0.80 to 0.95, reaching 46.79 AP85 versus 39.98 and 35.04. It localizes 500/900 targets at IoU 0.85, 32 more than UIU-Net.

On the 835 targets jointly detected by all four methods at IoU 0.50, mean absolute error across four box edges is 0.579 px for UIU-Net, 0.588 for TDIB, 0.617 for NS-FPN, and 0.618 for JFD³. TDIB ranks second on this common subset but localizes the most targets at IoU ≥ 0.85 across all 900. At comparable detection rates, it therefore turns more detections into tightly fitting boxes, the practical form of its strict-IoU gain. (Sec. 4.2, Fig. 6; Appendices 6.3 and 9.4, Table 20)

Reviewer_W6L1OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
1
Soundness
1
Read review
Paper summary

This paper reformulates the detection of degraded infrared small targets as "learning what to retain" rather than restoring a clean image, and proposes TDIB: using a frozen reversible prefix to limit information loss within the variational bottleneck. TDSR employs spatially weighted KL to shift the compression pressure towards the background and uses a fixed inverse transform to impose local fidelity constraints on the degraded input itself in the target region (without requiring a clean reference). IAR then transforms posterior consistency and drift discount ranking into supervision on the inference mean path. On the self-built synthetic benchmark IR-DegradeX, the macro AP reaches 68.19%, surpassing JFD³/NS-FPN, but the advantage is concentrated at strict IoU thresholds (AP50/AP75 still lags behind). Three sub-ablations support the division of labor among various components.

Critical strengths

1.The restatement of the problem is enlightening: it shifts the focus of degraded infrared small target detection from "repairing degradation/preserving all information" to "learning what to preserve", presenting a clear approach and distinguishing it from restoration-based methods. 2.The design of TDSR is targeted: freezing reversible prefixes to limit information loss within bottlenecks; spatially weighted KL to shift compression pressure from the target region to the background; local inverse fidelity takes the degraded input itself as a reference, without requiring clean images, and is only calculated in the target region, avoiding the forced retention of background and degradation. 3.IAR applies posterior changes to training: consistency constraints stabilize predictions, and drift discount sorting ranks candidates that are "well-positioned and stable under posterior perturbations" ahead of the mean path, while inference remains single-forward. 4.The parameter efficiency is good: TDIB has approximately 16.0M parameters, and its macro AP is higher than that of UIU-Net (50.5M) and ObjectBox (86.1M), which have more parameters.

Critical weaknesses
  1. The dataset adopted in this work is a self-constructed synthetic degradation benchmark. All seven degradation operators defined in IR-DegradeX are simulated degradations rather than data captured by real infrared sensors. All reported APs are obtained under synthetic degradation settings, so these results cannot be directly generalized to real-world infrared scenarios.
  2. Path-wise ablation studies are missing. The manuscript does not include comparison experiments such as training with only the mean path while discarding the sampling path, removing the detection loss associated with the sampling path, or migrating the local fidelity constraint to the mean path. As a result, the independent contribution of the noisy sampling path cannot be quantified, and the performance degradation when relying solely on the mean path remains unclear.
  3. The structural description of the detector is inadequate in the main text. Only function-level explanations are provided, while the layer-wise architecture, computational cost and full training configurations are deferred to the appendix. This weakens the reproducibility of the core method as presented in the main paper.
  4. The proposed diagnostic indicators lack cross-domain universality. Recon/σ/JS/Jitter/Agr are task-customized metrics specific to this work, which prevents fair horizontal comparison with results from other relevant studies.
  5. Hyperparameter sensitivity analysis is insufficient. Key coefficients including β, λ, τ, ρ and aea_e, together with their learning rate scheduling strategies, are only listed in the appendix, without systematic sensitivity investigation and discussion in the main body.
Questions for authors

1.Path ablation: At least including only following the mean path, removing the sampling path detection loss, changing to mean path calculation, and separately removing the path-level comparisons for consistency and ranking, to demonstrate whether the sampling path is irreplaceable. 2.Real degradation verification: Verification is conducted on real infrared sensor degradation data or real degradation collection scenarios, rather than solely evaluating on synthetic operators. 3.Cross-dataset validation: Report results on commonly used real-world infrared small target benchmarks such as SIRST, NUAA-SIRST, and IRSTD-1k to test generalization. 4.Explain and narrow the gap between AP50/AP75: Clarify the reasons why recovery-based methods excel at loose thresholds, and explain whether TDIB can compensate for this gap without sacrificing the advantage of strict thresholds. 5.Transparency of detector structure and cost: Provide layer-by-layer structure, parameter quantity, FLOPs, and inference latency in the main text or reproducible materials to facilitate fair comparison. 6.Robustness of freezing strategy: Analyze the impact of reversible prefix initialization quality, freezing timing, and whether partial fine-tuning is applied on the results. 7.Limitations of single deterministic path: Since inference only uses the mean path, it should be explicitly discussed why potential benefits such as multi-sample ensemble and uncertainty weighting are not adopted during testing, and to what extent does IAR compensate for the limitations of single path.

7 comments
AuthorsOpenReview ↗

Reply 1/3: Real degradation, established benchmarks, and sampled-path ablations

W1/Q2. Real infrared degradation. We added evaluation on IRBlurUAV. All four methods start from their IR-DegradeX LMix checkpoints and are adapted and selected using only synthetic IRBlurUAV data. The 4,118 real frames are used solely for testing. As in Table 1, JFD³ uses degraded-only supervision, without paired-clean losses.

MethodSynthetic AP / AP50 / AP75Real AP / AP50 / AP75
TDIB31.52 / 67.38 / 23.4115.00 / 48.99 / 3.85
JFD³23.49 / 57.98 / 12.9413.13 / 44.06 / 2.10
NS-FPN25.93 / 60.28 / 14.9912.41 / 41.06 / 3.04
UIU-Net19.26 / 54.63 / 6.959.03 / 35.87 / 1.18

TDIB leads in all three metrics on both the 2,880-image synthetic test set and all 4,118 real frames, by 5.59 AP and 1.87 AP over the strongest alternatives, respectively. This independent real-image test shows that the advantage of target-directed retention extends beyond the degradation operators used to develop IR-DegradeX. (Sec. 4.2, Table 2; Appendix 10.1)

Q3. Established infrared small-target benchmarks. To cover the review’s reference to “SIRST,” we evaluated both NUAA-SIRST and NUDT-SIRST, together with IRSTD-1K. Each method was adapted separately to each dataset and selected by validation AP. Entries are held-out box AP / AP50 (%).

MethodNUAA-SIRSTNUDT-SIRSTIRSTD-1K
TDIB40.29 / 89.5075.39 / 97.7136.33 / 83.06
JFD³44.16 / 90.5075.00 / 97.7636.77 / 80.99
NS-FPN41.10 / 87.5973.62 / 96.5639.83 / 85.75
UIU-Net44.95 / 81.2282.89 / 96.2035.27 / 76.55

No method wins AP on every dataset, but TDIB ranks second in AP50 on all three, within 1.00, 0.05, and 2.69 points of the best. Target boxes in these three benchmarks are much smaller than on IR-DegradeX (median areas: 24/25/49 versus 189 px²). On the 320 NUDT-SIRST targets ≤16 px², TDIB achieves the highest AP75 (91.6%) among the four methods. These results establish competitive transfer to established small-target benchmarks, with particularly strong strict localization on the smallest NUDT-SIRST targets. (Appendix 10.2, Tables 21–22)

W2/Q1. Contribution of the sampled path. We added the requested controls using a shared foundation and training protocol, with the same single mean-path inference. A3 (remove consistency and ranking) and A4 (remove ranking) already appeared in the original ablations and are explicitly identified in revised Table 3.

ConfigurationMacro AP (%)Δ vs. Full
Full TDIB68.187—
Mean path only67.863−0.324
Remove sampled-path detection loss67.792−0.395
Move local fidelity from sample to mean67.825−0.362
Remove consistency, retain ranking67.491−0.696
Remove ranking, retain consistency (A4)67.844−0.343
Remove both consistency and ranking (A3)67.913−0.274

Removing the entire sampled path costs 0.324 AP. Its objectives interact rather than contribute additively: removing sampled-path detection supervision costs 0.395 AP, and moving local fidelity to the mean path costs 0.362 AP. Sampled detection supervision trains recognition under posterior variation, while sampled-path fidelity protects local target structure under that variation. IAR further supervises the deployed mean path through prediction consistency and reliability ranking. Ranking adds 0.343 AP with consistency but loses 0.422 AP without it. Across three seeds, Full exceeds A3 by 0.31 AP and A4 by 0.39 AP, with no extra inference pass. None of these controls matches Full: sampled-path detection supervision, local fidelity, and IAR contribute jointly to the deployed mean-path detector. (Sec. 4.3, Tables 3–4; Appendices 11.1–11.2, Table 24)

AuthorsOpenReview ↗

Reply 2/3: Detector structure, localization, and sensitivity

W3/Q5. Detector structure and cost. Two invertible stages with Haar downsampling and coupling blocks map 3×512×512 → 12×256×256 → 48×128×128. A variational bottleneck follows the frozen prefix. A shared detector constructs fine features via a partial inverse, uses the selected latent at the middle scale, and encodes further for coarse features. Its 128-channel detection heads run at strides 2, 4, and 8. Every detection scale therefore depends on the selected representation. Inference uses only the posterior mean, without sampling or full reconstruction.

The model has 16.031M active parameters, 271.609 GFLOPs, and 21.73 ms median latency, versus 21.70 ms for the no-bottleneck A0 (RTX 4090, FP32, batch 1, 512×512). Appendix 8.1 (Tables 6–7) details the feature routes, module shapes, and loss-to-path assignments; Appendix 8.4 (Table 12) compares parameters, GFLOPs, and latency for all evaluated methods under one protocol. (Secs. 3.1 and 3.3; Appendices 8.1 and 8.4)

W4. Diagnostic versus benchmark metrics. Recon. and posterior σ\sigma characterize retention; classification JS, box jitter, and assignment agreement measure mean–sample prediction variation. They diagnose controlled TDIB variants, not cross-method ranking. Cross-method comparisons use held-out box AP/AP50/AP75, supplemented by AP85, precision/recall, detection probability, and false boxes/image. The revised Table 3 caption distinguishes test-set AP from validation diagnostics; Appendix 12 defines these diagnostics for controlled variants of one architecture, while Appendix 6.3 specifies the cross-method detection and localization measures. (Table 3; Appendices 6.3 and 12.1–12.4)

Q4. Loose versus strict localization. JFD³ and NS-FPN have higher AP50, and NS-FPN also leads at AP75 on IR-DegradeX. TDIB takes the lead at tighter overlap:

MethodAP50AP75AP85Correct @ IoU ≥ 0.85 (of 900)
TDIB98.0376.0246.79500
JFD³98.6676.7939.98459
NS-FPN98.4380.2435.04412

TDIB outperforms JFD³ and NS-FPN at every evaluated IoU threshold from 0.80 to 0.95. At IoU 0.85, it localizes 500/900 targets, versus 468 for UIU-Net, 459 for JFD³, and 412 for NS-FPN at validation-selected operating points.

Restoration and purification can strengthen degraded target responses, favoring approximate localization; TDIB instead constrains target-structure distortion through local inverse fidelity. The fidelity-placement ablation shows that the AP50/AP75 gap can be narrowed without losing strict-localization gains. Moving fidelity to the mean path raises AP50/AP75 from 98.03/76.02 to 98.61/77.22, while AP85 remains 45.78, still 5.80 points above JFD³ (39.98). Full retains higher AP85 (46.79) and Macro AP (68.19 vs. 67.83), so we keep sampled-path fidelity. On IRBlurUAV, TDIB also leads at AP50 and AP75 on synthetic and real tests. (Sec. 4.2, Fig. 6; Sec. 4.3, Table 4(a); Appendices 9.4 and 11.1)

W5. Hyperparameter sensitivity. We added 12 single-factor perturbations around the original default (68.19 Macro AP / 46.79 AP85), keeping the remaining training settings fixed. Here ρ\rho controls target-neighborhood relative KL pressure, the peak of βe\beta_e sets the maximum global KL weight, and aea_e jointly ramps sampling noise and the IAR losses.

Factor (default)Lower setting → Macro APHigher setting → Macro AP
Target KL ratio ρ\rho (0.10)0.05 → 68.250.20 → 68.01
Peak of βe\beta_e (0.01)0.005 → 67.990.02 → 67.93
Local fidelity λloc\lambda_{\mathrm{loc}} (1)0.5 → 67.982 → 68.32
Free-bits τ\tau (0.10)0.05 → 67.960.20 → 68.16
Joint IAR strength (1×)0.5× → 67.882× → 67.74
aea_e onset (epoch 8)epoch 4 → 67.56epoch 12 → 68.39

All 12 variants reach 67.56–68.39 AP, above JFD³ (66.84) and NS-FPN (66.73). The controls also reveal trade-offs: ρ=0.20\rho=0.20 raises AP85 to 48.10, while λloc=2\lambda_{\mathrm{loc}}=2 raises Macro AP to 68.32. The original defaults are retained. The global KL weight βe\beta_e ramps over epochs 4–11, while aea_e begins at epoch 8 and reaches full strength at epoch 19; onset changes retain the ramp duration and peak. We have added a compact sensitivity table to the main text and the full results to the appendix. (Sec. 4.3, Table 4(b); Appendix 11.3, Table 25; Appendix 8.2)

AuthorsOpenReview ↗

Reply 3/3: Foundation selection, fixed inverse, and single-pass inference

Q6. Foundation selection and prefix freezing. We compared foundations trained on L1, L2, L3, and LMix on the same L1–L3 validation subsets. Their Macro AP was 66.31, 67.11, 66.84, and 67.59, respectively, with LMix ranking first on each subset. During LMix pretraining, validation Macro AP was 60.87, 66.64, 67.59, and 67.12 at epochs 20, 50, 85, and 101. We therefore initialize TDIB from the validation-selected epoch-85 LMix checkpoint and freeze the invertible prefix before retention training begins. (Appendix 8.2, Table 8)

During retention training, only the bottleneck and detector are optimized. The fixed inverse maps the selected representation to the observation domain through the same transformation throughout training, allowing local fidelity to measure and constrain distortion introduced by the bottleneck. Fixing this reference mapping is therefore integral to TDSR's formulation rather than merely a training convenience. Delayed freezing or partial fine-tuning would allow the mapping to change during information selection. (Sec. 3.1; Appendix 7.2)

We further tested how these initialization and freezing choices affect final TDIB performance, using the same LMix retention data and otherwise matched training and evaluation settings. Both fine-tuning controls start from the epoch-85 LMix foundation and update only the last invertible stage.

SettingMacro AP (%)
TDIB: epoch-85 LMix foundation, frozen throughout68.19
Final-epoch (101) LMix foundation, frozen throughout68.12
L1 foundation, frozen throughout68.02
Delayed freezing: last stage frozen after 18K of 45K updates68.05
Partial fine-tuning: last stage updated throughout67.64

The final LMix checkpoint and the L1 foundation both remain within 0.17 AP of the default. With the same epoch-85 initialization, delayed freezing reduces AP by 0.14, while continuous partial fine-tuning reduces it by 0.55. The fully frozen default achieves the highest final AP among these five settings, supporting TDSR's fixed-inverse design while showing robustness to the tested initialization changes. All five settings also outperform the strongest baseline run (JFD³, 66.84 AP), showing that TDIB's advantage is not confined to the selected foundation checkpoint or freezing schedule.

Under the fixed-prefix configuration, the 12 coefficient and schedule perturbations also yield 67.56–68.39 Macro AP, showing stable retention training across these settings. (Appendix 11.3, Table 25)

Q7. Single-pass inference. IAR uses posterior perturbations during training to supervise the deterministic path deployed at test time. To test the added value of inference-time sampling, we averaged predictions from posterior draws using the same trained Full checkpoint:

InferenceMacro AP (%)Forward latency (ms)Relative cost
Posterior mean68.18721.731×
4 posterior samples68.42369.903.22×
8 posterior samples68.335133.906.16×
16 posterior samples68.385263.0512.11×

For this checkpoint, IAR improves single-pass AP by 0.27 over A3, whereas averaging four posterior samples adds 0.24 AP to Full at 3.22× the latency; 8 or 16 samples offer no further improvement. IAR already turns posterior variation into supervision for the mean-path detector, leaving little additional benefit for test-time sampling relative to its cost. Posterior σ is optimized for compression rather than calibrated as box-level uncertainty, so we do not use it for confidence-weighted test-time fusion. Inference therefore remains one deterministic forward pass. (Sec. 3.3; Appendix 11.4, Table 26; Appendix 8.4, Table 12)

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Response: Summary of Revisions and Additional Evidence

We thank all reviewers for their feedback and summarize below the evidence and revisions added during the rebuttal.

1. Generalization beyond IR-DegradeX (W6L1, r3xf, W3hB). TDIB achieves the highest absolute AP at each degradation level of IR-DegradeX. On IRBlurUAV, it leads the four compared methods in AP, AP50, and AP75 on both synthetic and real test sets, with AP margins of 5.59 and 1.87 over the strongest alternative. Adaptation and checkpoint selection use synthetic data only; all 4,118 real infrared frames are held out for testing.

On the established NUAA-SIRST, NUDT-SIRST, and IRSTD-1K benchmarks, TDIB ranks second in AP50 on all three. Following W3hB's suggestion, we evaluated the existing dataset-adapted checkpoints on test images degraded at L1–L3, without further training or recalibration. Averaged equally over the three benchmarks and three degradation levels, TDIB achieves the highest mean AP50 (79.99, +1.43 points over the next best) and the second-highest mean AP (0.72 points behind the best). It leads in both AP and AP50 on degraded NUDT-SIRST (56.35 / 89.04) and has the smallest AP drop on degraded NUAA-SIRST and IRSTD-1K (6.30 and 5.75 points). These tests measure robustness to added test-time degradation, distinct from the degradation-aware training or adaptation in the main evaluations. Across the seven IR-DegradeX operators, TDIB also achieves the highest mean and worst-operator AP (68.22 and 64.61).

2. Complete-method comparison and contribution of retention (r3xf). The requested InvDet comparison shows a 25.34 Macro AP advantage for TDIB's complete design, which differs in information selection, detection architecture, and training objectives. The shared-foundation Full/A0 control isolates the contribution of the proposed retention framework within the same detector: +0.88 mean Macro AP and +3.30 mean AP85 across three matched seeds, with a positive AP85 gain in every run. This is controlled evidence that learning what to preserve improves precise localization even when a recoverable representation is already available.

3. Robustness and practical meaning of precise localization (W6L1, r3xf). Across three seeds, every TDIB run exceeds every run of JFD³ and NS-FPN, with mean margins of 1.69 and 1.79 AP. TDIB also outperforms both at every IoU threshold from 0.80 to 0.95 (AP85: 46.79 vs. 39.98 and 35.04). At validation-selected operating points, the four methods have similar center-based detection rates (95.6–97.2%), yet TDIB localizes 500 of 900 targets at IoU ≥ 0.85, 32 more than the next best method. The strict-IoU advantage thus corresponds to more accurately localized targets, not simply more detections.

4. Mechanism and training choices (W6L1, W3hB). No control that removes or relocates part of the sampled path matches Full, and ranking adds 0.34 AP on top of consistency. Replacing drift-discounted reliability with localization quality alone lowers three-seed mean AP from 68.20 to 67.80. On 740 matched target/background patch pairs, background patches are reconstructed with an MAE of 0.136 gray levels, while retained clutter such as date characters receives a detection score of only 0.0044. In foundation and freezing controls, both alternative frozen initializations stay within 0.17 AP of the default, whereas delayed freezing and continuous partial fine-tuning reduce AP by 0.14 and 0.55, supporting the fixed inverse used by TDSR. All 12 coefficient and schedule perturbations yield 67.56–68.39 Macro AP, and averaging four posterior samples adds only 0.24 AP at 3.22× latency, supporting single-pass inference.

The revised manuscript (changes highlighted) adds InvDet to Table 1, IRBlurUAV as Table 2, and path controls and sensitivity as Table 4, together with Appendix 10 (External Benchmark Evaluation), Appendix 11 (Additional Controlled Analyses), architecture details, and inference-cost measurements. The further degraded-benchmark and freezing experiments are reported in our discussion responses. Together, these results strengthen the central claim: learning what to preserve from a recoverable representation improves precise localization under degradation, with supporting evidence extending to real infrared imagery and established small-target benchmarks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.