- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work introduces InstaPoint, an instance‑adaptive supervision‑assignment framework for point‑supervised oriented object detection in remote sensing images. Motivated by the limitation of prior patch‑wise supervision selection that fails to handle intra‑patch spatial heterogeneity where isolated and crowded objects co‑occur, InstaPoint adopts a two‑step pipeline: Instance Density Calibration (IDC delivers prediction‑agnostic initial assignments based on class‑calibrated local point spacing during warm‑up, followed by Instance‑wise Supervision Assignment (ISA), which performs a one‑shot permanent reassignment per instance by picking either SAM‑ or watershed‑derived masks via Gaussian Wasserstein distance against current detector predictions. Built upon existing detectors without architectural modification, InstaPoint yields overall AP50 gains over strong baselines on DOTA‑v1.0 and DIOR‑R under both end‑to‑end and two‑stage pseudo‑labeling settings, though improvements remain incremental and the method still relies heavily on external mask priors.
Critical strengths
- The paper identifies a real limitation of prior patch‑wise supervision assignment: patch‑level heuristics cannot handle mixed dense/isolated instances within one patch. Moving to instance‑level assignment is a meaningful conceptual direction.
- The framework is plug‑and‑play and does not modify the underlying detector architecture.
- Evaluations are conducted on two standard remote sensing benchmarks with both end‑to‑end and two‑stage pseudo‑label settings, and appendix provides reassignment statistics.
Critical weaknesses
- ISA locks supervision sources permanently based on early noisy model predictions, which may create harmful feedback loops. This risk is neither sufficiently discussed nor empirically evaluated.
- The paper lacks two key baselines: always‑SAM and always‑watershed for all instances. It remains unclear whether performance gains stem from adaptive instance‑wise selection or merely mixing two mask sources.
- Several classes show degraded AP50 compared with baselines. No category‑level attribution is provided to link drops to IDC, ISA or mask quality, which hurts the method’s demonstrated reliability.
- Performance strongly depends on when ISA is activated. The paper only reports the best epoch but does not investigate this instability or compare against more robust alternatives such as iterative dynamic reassignment. The synergy between IDC initialization and ISA is under‑analyzed.
- Appendix Table4 reports supervision‑source switching ratios but does not verify whether isolated instances tend to select SAM and dense instances select watershed, as the paper hypothesizes.
- Additional training‑time compute and GPU memory introduced by ISA are not reported; inference‑side cost is also unclear.
Questions for authors
- There is numerical inconsistency between the ISA‑only setting in Table3(a) and epoch‑0 ISA activation in Table3(c), which are logically equivalent. Could you explain this discrepancy?
- Please add ablations for two extreme baselines: always‑SAM and always‑watershed for all instances, to disentangle the source of performance gains.
- Some categories suffer AP50 degradation (e.g. PL, GTF, RA, BC, BR, TS). Please provide category‑wise analysis to attribute these drops.
- Could you provide statistics linking ISA reassignment results to instance spatial density (s_i), to validate your core hypothesis that isolated instances prefer SAM and dense instances prefer watershed?
- Please report ISA’s extra training‑time computation and GPU memory overhead, and clarify whether inference incurs any additional cost.
7 comments
Response to Reviewer RvAw (2/3)
W3–Q3. Per-category performance degradation
We clarify that GTF improves over Point2RBox-v3 in both protocols on both datasets (Tables 1–2). For the remaining decreases, Fig. 5 shows improved mean target IoU for DOTA PL and BR with the GT orientation fixed, despite their lower detection AP. Differences in predicted orientation, confidence ranking, and missed or false detections may contribute to this discrepancy, as these factors are not captured by the extent-quality diagnostic.
We further report category-wise selection accuracy and mean target IoU in the table below. For both DOTA-v1.0 and DIOR-R, we use candidates and predicted orientations recorded at ISA selection.
| Dataset | Class | Selection accuracy (%) | Mean target IoU (%) |
|---|---|---|---|
| DOTA-v1.0 | Plane | 51.4 → 68.5 | 69.3 → 74.2 |
| DOTA-v1.0 | Bridge | 48.7 → 82.7 | 26.6 → 34.5 |
| DOTA-v1.0 | Roundabout | 69.0 → 72.3 | 54.5 → 55.6 |
| DIOR-R | Basketball court | 52.6 → 88.7 | 57.2 → 72.7 |
| DIOR-R | Train station | 28.8 → 56.9 | 31.6 → 41.4 |
Arrows denote IDC ISA. Selection accuracy measures how often the higher-GT-IoU candidate is selected, excluding ties; mean target IoU measures the average overlap of the selected candidates with the GT RBoxes. For DOTA bridge, even GT-based selection of the better candidate reaches only 37.4% mean target IoU, indicating limited candidate quality. These results show that ISA selects more geometrically accurate supervision than IDC in every listed category. The qualitative comparisons in Appendix A.4 (Fig. 6) complement these diagnostics, showing improved localization for basketball court and train station on DIOR-R and for plane and bridge on DOTA-v1.0.
W4. Sensitivity to ISA activation and initialization
Table 3(c) of the submitted manuscript, now Table 4(c) in the revision, evaluates warm-up durations and . Across these settings, E2E/FCOS ranges over 40.41–42.49/45.35–47.10 on DIOR-R and 58.88–62.53/65.11–67.74 on DOTA-v1.0. The module ablation (submitted Table 3(a), revised Table 4(a)) also compares baseline and IDC warm-up before ISA. We further added single-source warm-up experiments and repeated reassignment schedules.
| Training setup | E2E | Two-stage (FCOS) |
|---|---|---|
| Baseline warm-up + ISA (6 warm-up epochs; ISA starts at epoch 7) | 60.23 | 66.01 |
| Watershed warm-up + ISA (6 warm-up epochs; ISA starts at epoch 7) | 60.77 | 65.89 |
| SAM warm-up + ISA (6 warm-up epochs; ISA starts at epoch 7) | 61.80 | 67.61 |
| ISA at epochs 7, 9, and 11 | 60.90 | 66.37 |
| ISA at every epoch from 7 to 12 | 61.20 | 67.01 |
| IDC warm-up + ISA (ours) (6 warm-up epochs; ISA starts at epoch 7) | 62.53 | 67.74 |
IDC warm-up followed by ISA achieves the highest performance among the evaluated settings, improving E2E/FCOS by 2.30/1.73 points over baseline warm-up followed by ISA. Neither repeated-reassignment schedule improves over the proposed one-time schedule, despite requiring additional computation.
Response to Reviewer RvAw (3/3)
W5–Q4. Relationship between density and ISA reassignment
To examine how ISA assignments relate to local spacing, we group DOTA-v1.0 instances by : the distance to the nearest same-class point divided by the class-specific spacing reference. Smaller values indicate closer same-class neighbors. Using the candidate RBoxes and predicted orientations recorded at ISA selection, we report initial IDC assignments, ISA transitions, and GT-evaluated selection accuracy (%).
| Proportion (%) | IDC init. | ISA switch / keep | Final SAM (%) | Acc. (IDC → ISA) | |
|---|---|---|---|---|---|
| 0–0.5 | 19.0 | WS | W→S 33.8% / 66.2% | 33.8 | 65.0 → 82.3 |
| 0.5–1 | 45.9 | WS | W→S 42.4% / 57.6% | 42.4 | 52.0 → 75.2 |
| 1–2 | 20.6 | SAM | S→W 50.0% / 50.0% | 50.0 | 53.1 → 74.9 |
| 2–4 | 7.9 | SAM | S→W 48.3% / 51.7% | 51.7 | 56.0 → 75.6 |
| $ > 4$ | 4.3 | SAM | S→W 44.8% / 55.2% | 55.2 | 58.9 → 77.8 |
| No same-class neighbor | 2.3 | SAM | S→W 40.5% / 59.5% | 59.5 | 65.4 → 77.3 |
The final SAM proportion increases monotonically from 33.8% to 55.2% across the finite spacing intervals and reaches 59.5% for instances without a same-class neighbor. WS (or W) and S denote watershed and SAM, respectively. Instances without a same-class neighbor in the input patch are listed separately and initialized with SAM by IDC. IDC accuracy is higher in the very dense () and isolated (>4) regimes than near the decision boundary (), supporting the use of normalized spacing as a meaningful statistical prior. At the same time, substantial switching occurs across all intervals—most notably near the boundary—and ISA consistently improves selection accuracy in every spacing regime. Overall, ISA changes 42.9% of assignments and improves selection accuracy from 55.6% to 76.7%, correcting instance-specific exceptions that spacing alone cannot resolve.
W6–Q5. Training overhead and inference cost
The additional computation comes from generating and comparing candidate extent targets during training. We measured computational costs under the same hardware setup (a single RTX 4090 with batch size 2). SAM caching means precomputing and reusing SAM masks during training.
| Method | Preprocessing | E2E training time | Peak allocated GPU memory |
|---|---|---|---|
| Point2RBox-v3 (baseline) (reproduced) | None | 10 h 19 min | ≈7.3 GB |
| InstaPoint, with SAM caching | IDC: min; SAM cache: 1 h 11 min | 18 h 39 min | ≈7.3 GB |
| InstaPoint, without caching | IDC: min | 27 h 22 min | ≈7.3 GB |
With SAM caching, we separately profiled the ISA selection and fixed-source phases:
| Training phase | Time per iteration (s) | Peak allocated GPU memory (MB) |
|---|---|---|
| ISA selection | 1.279 | 6,693 |
| Fixed-source phase | 0.910 | 6,972 |
The time difference corresponds to about 39 additional minutes over 6,400 iterations in the single ISA selection epoch. The overhead primarily comes from constructing candidate extent targets. The cache occupies 182 MB. Including cache generation, the cached pipeline takes approximately 19 h 50 min. Peak allocated GPU memory, recorded in the training logs, is similar across the three settings (approximately 7.3 GB). All three settings use the same detector, and candidate targets are detached from the gradient computation, which is consistent with their similar measured memory use.
Both candidates are compared only at the designated reassignment stage. Once the source is fixed, subsequent iterations recompute the selected extent target using the current predicted orientation without repeating the two-source comparison. At inference, the detector architecture remains unchanged, with no additional ISA modules or parameters.
Note that no mask generation or supervision-source assignment is performed at inference.
Q1. Difference between ISA-only and epoch-0 activation
The label "ISA-only" did not make its warm-up setting clear. ISA-only in Table 3(a) of the submitted manuscript uses the baseline assignment for six warm-up epochs and activates ISA at epoch 7, whereas in Table 3(c) of the submitted manuscript activates ISA from the start without warm-up. Table 5(c) of the revised manuscript explicitly lists the warm-up source and duration. We have clarified the notation in revised Table 4(c) and the main text: denotes the number of IDC warm-up epochs, and ISA starts at epoch .
We have incorporated the additional results and analyses presented above, along with the corresponding clarifications, into the revised manuscript and supplementary material.
Response to Reviewer RvAw (1/3)
We thank the reviewer for the careful review and constructive feedback. We address the comments below.
W1. Risk of reinforcing noisy initial supervision
During IDC warm-up, supervision sources are assigned independently of detector predictions. ISA subsequently compares the two candidate extent targets—the alternative width-height targets derived from SAM and watershed masks—with the detector’s extent predictions. The source choice is then fixed, so later predictions do not trigger further source reassignment.
Table 3(a) of the submitted manuscript, now Table 4(a) in the revision, includes IDC-only and baseline warm-up + ISA. We additionally test watershed-only and SAM-only warm-up. Baseline warm-up uses Point2RBox-v3’s patch-wise assignment, while watershed (WS) and SAM warm-up use the indicated source for every instance. For every setting with ISA, the detector uses the indicated source for six warm-up epochs before ISA starts at epoch 7. Point2RBox-v3 uses its published AP50 values.
Compared with using the same source throughout training, ISA improves E2E/two-stage AP50 by 3.41/2.34 points after watershed warm-up and 0.91/0.99 points after SAM warm-up. IDC warm-up achieves the best performance in both protocols. IDC-only retains a spacing-based prior throughout training, but spacing alone cannot resolve differences in candidate quality. Using IDC only for initialization lets ISA subsequently revise those assignments using image-conditioned predictions. The 2.30/1.73-point gain over baseline warm-up followed by ISA supports the complementary roles of IDC and ISA.
We evaluate the candidate rotated boxes (RBoxes) and predicted orientations recorded when ISA fixes assignments after IDC warm-up. Selection accuracy measures how often the candidate with higher GT IoU is selected. ISA corrects 72.0% of initially incorrect assignments, while changing 19.6% of initially correct assignments to the inferior candidate. Overall selection accuracy increases from 55.6% to 76.7%. Although some incorrect assignments remain, ISA corrects a substantial fraction of the initial errors.
To examine whether ISA remains effective when the detector’s predictions are less accurate, we group instances by prediction IoU: the overlap between the detector’s prediction and the GT RBox at ISA selection. GT is used only for evaluation.
At selection, 91.8% of instances have a prediction IoU of at least 0.5. ISA’s selection accuracy is 72.4% for the 0.5–0.7 group and 79.2% for the ≥0.7 group.
W2–Q2. Always-SAM and always-watershed baselines
We added both single-source baselines using the same implementation and candidate-processing procedure.
InstaPoint outperforms both single-source baselines in both protocols.
To examine whether the benefit comes from which instances receive each source, we evaluated random mixing applied after six epochs of IDC warm-up. It shuffles ISA’s source assignments within each class, preserves the per-class SAM counts, and keeps the shuffled assignments fixed for the remaining epochs. InstaPoint outperforms random mixing in both protocols, supporting the benefit of instance-specific source selection beyond using the two sources in the same proportions.
We also reverse the ISA criterion by selecting the candidate with the larger size loss instead of the smaller one. This reduces E2E/FCOS AP50 to 59.55/64.92 on DOTA-v1.0, supporting prediction–supervision consistency as a useful assignment signal.