acceptodds
Under review as a conference paper at ICLR 2027

Different Instances, Different Supervision: Instance-Adaptive Point-Supervised Oriented Object Detection

Abstract

Point annotations reduce labeling costs for oriented object detection but provide neither object scale nor orientation. Complementary mask-based supervision sources provide extent cues, yet their relative reliability varies with the local spatial configuration of each instance. A patch-wise source choice can therefore be unsuitable for individual instances when isolated and densely clustered objects coexist in remote-sensing imagery. We find that local spatial configuration provides an effective cue for instance-level supervision selection, yet spatial cues alone cannot fully determine which source is most suitable for each instance. Based on this observation, we propose InstaPoint, an instance-adaptive framework that combines spatially guided initialization with prediction-guided supervision assignment. Instance Density Calibration (IDC) provides prediction-independent instance-specific initial assignments, while Instance-wise Supervision Assignment (ISA) subsequently selects, for each instance, the candidate supervision most consistent with the detector’s current prediction and retains the selected source for the remainder of training. ISA revises the initial supervision source for 47.62% of instances on DIOR-R and 42.99% on DOTA-v1.0, indicating substantial instance-wise reassignment beyond the spatial initialization. InstaPoint changes how extent supervision is assigned while retaining the baseline detector architecture and other training objectives. Under the end-to-end and two-stage pseudo-labeling protocols, InstaPoint achieves 42.49/47.10 on DIOR-R and 62.53/67.74 on DOTA-v1.0, surpassing the strongest point-supervised baseline by 0.79/0.50 points on DIOR-R and 2.92/1.65 points on DOTA-v1.0.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
weakly supervised learning, oriented object detection, point supervision, instance-adaptive supervision, remote sensing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RvAwOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work introduces InstaPoint, an instance‑adaptive supervision‑assignment framework for point‑supervised oriented object detection in remote sensing images. Motivated by the limitation of prior patch‑wise supervision selection that fails to handle intra‑patch spatial heterogeneity where isolated and crowded objects co‑occur, InstaPoint adopts a two‑step pipeline: Instance Density Calibration (IDC delivers prediction‑agnostic initial assignments based on class‑calibrated local point spacing during warm‑up, followed by Instance‑wise Supervision Assignment (ISA), which performs a one‑shot permanent reassignment per instance by picking either SAM‑ or watershed‑derived masks via Gaussian Wasserstein distance against current detector predictions. Built upon existing detectors without architectural modification, InstaPoint yields overall AP50 gains over strong baselines on DOTA‑v1.0 and DIOR‑R under both end‑to‑end and two‑stage pseudo‑labeling settings, though improvements remain incremental and the method still relies heavily on external mask priors.

Critical strengths
  1. The paper identifies a real limitation of prior patch‑wise supervision assignment: patch‑level heuristics cannot handle mixed dense/isolated instances within one patch. Moving to instance‑level assignment is a meaningful conceptual direction.
  2. The framework is plug‑and‑play and does not modify the underlying detector architecture.
  3. Evaluations are conducted on two standard remote sensing benchmarks with both end‑to‑end and two‑stage pseudo‑label settings, and appendix provides reassignment statistics.
Critical weaknesses
  1. ISA locks supervision sources permanently based on early noisy model predictions, which may create harmful feedback loops. This risk is neither sufficiently discussed nor empirically evaluated.
  2. The paper lacks two key baselines: always‑SAM and always‑watershed for all instances. It remains unclear whether performance gains stem from adaptive instance‑wise selection or merely mixing two mask sources.
  3. Several classes show degraded AP50 compared with baselines. No category‑level attribution is provided to link drops to IDC, ISA or mask quality, which hurts the method’s demonstrated reliability.
  4. Performance strongly depends on when ISA is activated. The paper only reports the best epoch but does not investigate this instability or compare against more robust alternatives such as iterative dynamic reassignment. The synergy between IDC initialization and ISA is under‑analyzed.
  5. Appendix Table4 reports supervision‑source switching ratios but does not verify whether isolated instances tend to select SAM and dense instances select watershed, as the paper hypothesizes.
  6. Additional training‑time compute and GPU memory introduced by ISA are not reported; inference‑side cost is also unclear.
Questions for authors
  1. There is numerical inconsistency between the ISA‑only setting in Table3(a) and epoch‑0 ISA activation in Table3(c), which are logically equivalent. Could you explain this discrepancy?
  2. Please add ablations for two extreme baselines: always‑SAM and always‑watershed for all instances, to disentangle the source of performance gains.
  3. Some categories suffer AP50 degradation (e.g. PL, GTF, RA, BC, BR, TS). Please provide category‑wise analysis to attribute these drops.
  4. Could you provide statistics linking ISA reassignment results to instance spatial density (s_i), to validate your core hypothesis that isolated instances prefer SAM and dense instances prefer watershed?
  5. Please report ISA’s extra training‑time computation and GPU memory overhead, and clarify whether inference incurs any additional cost.
7 comments
AuthorsOpenReview ↗

Response to Reviewer RvAw (1/3)

We thank the reviewer for the careful review and constructive feedback. We address the comments below.

W1. Risk of reinforcing noisy initial supervision

During IDC warm-up, supervision sources are assigned independently of detector predictions. ISA subsequently compares the two candidate extent targets—the alternative width-height targets derived from SAM and watershed masks—with the detector’s extent predictions. The source choice is then fixed, so later predictions do not trigger further source reassignment.

Table 3(a) of the submitted manuscript, now Table 4(a) in the revision, includes IDC-only and baseline warm-up + ISA. We additionally test watershed-only and SAM-only warm-up. Baseline warm-up uses Point2RBox-v3’s patch-wise assignment, while watershed (WS) and SAM warm-up use the indicated source for every instance. For every setting with ISA, the detector uses the indicated source for six warm-up epochs before ISA starts at epoch 7. Point2RBox-v3 uses its published AP50\mathrm{AP}_{50} values.

SettingBaseline init.WS init.SAM init.IDC init.ISAE2E AP50\mathrm{AP}_{50}FCOS AP50\mathrm{AP}_{50}
Point2RBox-v3 (baseline)✓59.6166.09
Always-watershed✓57.3663.55
Always-SAM✓60.8966.62
IDC-only✓58.8866.00
Baseline warm-up + ISA✓✓60.2366.01
Watershed warm-up + ISA✓✓60.7765.89
SAM warm-up + ISA✓✓61.8067.61
InstaPoint (full)✓✓62.5367.74

Compared with using the same source throughout training, ISA improves E2E/two-stage AP50\mathrm{AP}_{50} by 3.41/2.34 points after watershed warm-up and 0.91/0.99 points after SAM warm-up. IDC warm-up achieves the best performance in both protocols. IDC-only retains a spacing-based prior throughout training, but spacing alone cannot resolve differences in candidate quality. Using IDC only for initialization lets ISA subsequently revise those assignments using image-conditioned predictions. The 2.30/1.73-point gain over baseline warm-up followed by ISA supports the complementary roles of IDC and ISA.

We evaluate the candidate rotated boxes (RBoxes) and predicted orientations recorded when ISA fixes assignments after IDC warm-up. Selection accuracy measures how often the candidate with higher GT IoU is selected. ISA corrects 72.0% of initially incorrect assignments, while changing 19.6% of initially correct assignments to the inferior candidate. Overall selection accuracy increases from 55.6% to 76.7%. Although some incorrect assignments remain, ISA corrects a substantial fraction of the initial errors.

To examine whether ISA remains effective when the detector’s predictions are less accurate, we group instances by prediction IoU: the overlap between the detector’s prediction and the GT RBox at ISA selection. GT is used only for evaluation.

Prediction IoUProportion (%)IDC accuracy (%)ISA accuracy (%)
$ < 0.3$2.651.356.5
0.30.3–0.50.55.656.173.3
0.50.5–0.70.723.953.072.4
≥0.7\geq0.767.856.679.2

At selection, 91.8% of instances have a prediction IoU of at least 0.5. ISA’s selection accuracy is 72.4% for the 0.5–0.7 group and 79.2% for the ≥0.7\geq0.7 group.

W2–Q2. Always-SAM and always-watershed baselines

We added both single-source baselines using the same implementation and candidate-processing procedure.

MethodE2E AP50\mathrm{AP}_{50}Two-stage (FCOS) AP50\mathrm{AP}_{50}
Always-watershed57.3663.55
Always-SAM60.8966.62
Random mixing after IDC warm-up60.7166.91
Reversed ISA (larger size loss)59.5564.92
InstaPoint (ours)62.5367.74

InstaPoint outperforms both single-source baselines in both protocols.

To examine whether the benefit comes from which instances receive each source, we evaluated random mixing applied after six epochs of IDC warm-up. It shuffles ISA’s source assignments within each class, preserves the per-class SAM counts, and keeps the shuffled assignments fixed for the remaining epochs. InstaPoint outperforms random mixing in both protocols, supporting the benefit of instance-specific source selection beyond using the two sources in the same proportions.

We also reverse the ISA criterion by selecting the candidate with the larger size loss instead of the smaller one. This reduces E2E/FCOS AP50\mathrm{AP}_{50} to 59.55/64.92 on DOTA-v1.0, supporting prediction–supervision consistency as a useful assignment signal.

AuthorsOpenReview ↗

Response to Reviewer RvAw (2/3)

W3–Q3. Per-category performance degradation

We clarify that GTF improves over Point2RBox-v3 in both protocols on both datasets (Tables 1–2). For the remaining decreases, Fig. 5 shows improved mean target IoU for DOTA PL and BR with the GT orientation fixed, despite their lower detection AP. Differences in predicted orientation, confidence ranking, and missed or false detections may contribute to this discrepancy, as these factors are not captured by the extent-quality diagnostic.

We further report category-wise selection accuracy and mean target IoU in the table below. For both DOTA-v1.0 and DIOR-R, we use candidates and predicted orientations recorded at ISA selection.

DatasetClassSelection accuracy (%)Mean target IoU (%)
DOTA-v1.0Plane51.4 → 68.569.3 → 74.2
DOTA-v1.0Bridge48.7 → 82.726.6 → 34.5
DOTA-v1.0Roundabout69.0 → 72.354.5 → 55.6
DIOR-RBasketball court52.6 → 88.757.2 → 72.7
DIOR-RTrain station28.8 → 56.931.6 → 41.4

Arrows denote IDC →\rightarrow ISA. Selection accuracy measures how often the higher-GT-IoU candidate is selected, excluding ties; mean target IoU measures the average overlap of the selected candidates with the GT RBoxes. For DOTA bridge, even GT-based selection of the better candidate reaches only 37.4% mean target IoU, indicating limited candidate quality. These results show that ISA selects more geometrically accurate supervision than IDC in every listed category. The qualitative comparisons in Appendix A.4 (Fig. 6) complement these diagnostics, showing improved localization for basketball court and train station on DIOR-R and for plane and bridge on DOTA-v1.0.

W4. Sensitivity to ISA activation and initialization

Table 3(c) of the submitted manuscript, now Table 4(c) in the revision, evaluates warm-up durations T=0,3,6,9,T = 0, 3, 6, 9, and 1212. Across these settings, E2E/FCOS AP50\mathrm{AP}_{50} ranges over 40.41–42.49/45.35–47.10 on DIOR-R and 58.88–62.53/65.11–67.74 on DOTA-v1.0. The module ablation (submitted Table 3(a), revised Table 4(a)) also compares baseline and IDC warm-up before ISA. We further added single-source warm-up experiments and repeated reassignment schedules.

Training setupE2E AP50\mathrm{AP}_{50}Two-stage (FCOS) AP50\mathrm{AP}_{50}
Baseline warm-up + ISA (6 warm-up epochs; ISA starts at epoch 7)60.2366.01
Watershed warm-up + ISA (6 warm-up epochs; ISA starts at epoch 7)60.7765.89
SAM warm-up + ISA (6 warm-up epochs; ISA starts at epoch 7)61.8067.61
ISA at epochs 7, 9, and 1160.9066.37
ISA at every epoch from 7 to 1261.2067.01
IDC warm-up + ISA (ours) (6 warm-up epochs; ISA starts at epoch 7)62.5367.74

IDC warm-up followed by ISA achieves the highest performance among the evaluated settings, improving E2E/FCOS AP50\mathrm{AP}_{50} by 2.30/1.73 points over baseline warm-up followed by ISA. Neither repeated-reassignment schedule improves over the proposed one-time schedule, despite requiring additional computation.

AuthorsOpenReview ↗

Response to Reviewer RvAw (3/3)

W5–Q4. Relationship between density and ISA reassignment

To examine how ISA assignments relate to local spacing, we group DOTA-v1.0 instances by si=di/ℓcis_i = d_i / \ell_{c_i}: the distance to the nearest same-class point divided by the class-specific spacing reference. Smaller values indicate closer same-class neighbors. Using the candidate RBoxes and predicted orientations recorded at ISA selection, we report initial IDC assignments, ISA transitions, and GT-evaluated selection accuracy (%).

sis_iProportion (%)IDC init.ISA switch / keepFinal SAM (%)Acc. (IDC → ISA)
0–0.519.0WSW→S 33.8% / 66.2%33.865.0 → 82.3
0.5–145.9WSW→S 42.4% / 57.6%42.452.0 → 75.2
1–220.6SAMS→W 50.0% / 50.0%50.053.1 → 74.9
2–47.9SAMS→W 48.3% / 51.7%51.756.0 → 75.6
$ > 4$4.3SAMS→W 44.8% / 55.2%55.258.9 → 77.8
No same-class neighbor2.3SAMS→W 40.5% / 59.5%59.565.4 → 77.3

The final SAM proportion increases monotonically from 33.8% to 55.2% across the finite spacing intervals and reaches 59.5% for instances without a same-class neighbor. WS (or W) and S denote watershed and SAM, respectively. Instances without a same-class neighbor in the input patch are listed separately and initialized with SAM by IDC. IDC accuracy is higher in the very dense (0–0.50\text{--}0.5) and isolated (>4) regimes than near the decision boundary (0.5–20.5\text{--}2), supporting the use of normalized spacing as a meaningful statistical prior. At the same time, substantial switching occurs across all intervals—most notably near the boundary—and ISA consistently improves selection accuracy in every spacing regime. Overall, ISA changes 42.9% of assignments and improves selection accuracy from 55.6% to 76.7%, correcting instance-specific exceptions that spacing alone cannot resolve.

W6–Q5. Training overhead and inference cost

The additional computation comes from generating and comparing candidate extent targets during training. We measured computational costs under the same hardware setup (a single RTX 4090 with batch size 2). SAM caching means precomputing and reusing SAM masks during training.

MethodPreprocessingE2E training timePeak allocated GPU memory
Point2RBox-v3 (baseline) (reproduced)None10 h 19 min≈7.3 GB
InstaPoint, with SAM cachingIDC: ≤1\leq 1 min; SAM cache: 1 h 11 min18 h 39 min≈7.3 GB
InstaPoint, without cachingIDC: ≤1\leq 1 min27 h 22 min≈7.3 GB

With SAM caching, we separately profiled the ISA selection and fixed-source phases:

Training phaseTime per iteration (s)Peak allocated GPU memory (MB)
ISA selection1.2796,693
Fixed-source phase0.9106,972

The time difference corresponds to about 39 additional minutes over 6,400 iterations in the single ISA selection epoch. The overhead primarily comes from constructing candidate extent targets. The cache occupies 182 MB. Including cache generation, the cached pipeline takes approximately 19 h 50 min. Peak allocated GPU memory, recorded in the training logs, is similar across the three settings (approximately 7.3 GB). All three settings use the same detector, and candidate targets are detached from the gradient computation, which is consistent with their similar measured memory use.

Both candidates are compared only at the designated reassignment stage. Once the source is fixed, subsequent iterations recompute the selected extent target using the current predicted orientation without repeating the two-source comparison. At inference, the detector architecture remains unchanged, with no additional ISA modules or parameters.

Note that no mask generation or supervision-source assignment is performed at inference.

Q1. Difference between ISA-only and epoch-0 activation

The label "ISA-only" did not make its warm-up setting clear. ISA-only in Table 3(a) of the submitted manuscript uses the baseline assignment for six warm-up epochs and activates ISA at epoch 7, whereas T=0T=0 in Table 3(c) of the submitted manuscript activates ISA from the start without warm-up. Table 5(c) of the revised manuscript explicitly lists the warm-up source and duration. We have clarified the notation in revised Table 4(c) and the main text: TT denotes the number of IDC warm-up epochs, and ISA starts at epoch T+1T+1.


We have incorporated the additional results and analyses presented above, along with the corresponding clarifications, into the revised manuscript and supplementary material.

Reviewer_P9skOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses point-supervised oriented object detection (OOD) in remote sensing, where only point annotation are available and object orientation and structure must be recovered. Buildig on Point2Box-v3, which combine the complementry SAM and watershed masks but assign them in a patch-wise manner accoring to the instance count, the author observe that this is mismatched to hetrogeneous scenes contain both isolated and densely clustered objects. They propose InstaPoint, an instance adaptive framework with two stage: (1) Instance Density Calibration (IDC) performs predcition independently initialize by assigning SAM or watershed to each instance according to class-calibrated local same-class spacing. (2) After a fixed warm up, Instance-wise Supervision Assignment (ISA) perform one time reassignment by selecting the candidate extent target that yield the lower Gaussian Wasserstein Distance loss relative to detector's current prediction. The chosen source is then retained for the rest of training. The method leaves the underlying detector architecture and remaining losses unchanged.

Critical strengths
  • The paper identify a concrete weakness of patch-wise source selection. Shifting the assignment granularity to the individual instance and combining a prediction-free spatial prior (IDC) with a one-time prediction-guided refinement (ISA) is a simple, well-motivated solution that directly addresses the identified limitation of prior work.

  • The reported gains are positive on both dataset and both evaluation settings. These results suggest that instance-adaptive supervision may be particularly useful in scenes with dense and heterogeneous spatial layouts.

Critical weaknesses
  • As in Equation 7, IDC uses class-calibrated nearest neighbour spacing for selecting source assignment, SAM for isolated instances and watershed for closely spaced instances. This is sensible, but spatial distance between annotation points does not establish whether a SAM mask or watershed region better represents an object extent. Two objects with identical point spacing may differ in size, aspect ratio, contrast, orientation, and boundary visibility. A large isolated object can also have a different segmentation difficulty from a small isolated object. ( For example, clustered objects blending into a similar colored background)

  • ISA selects between SAM and watershed extent targets according to which produces a lower GWD loss relative to the detector's current prediction. However, a lower loss relative to the current prediction does not necessarily indicate that the selected target better represents the true object geometry. If the detector's prediction is inaccurate, ISA may favor a candidate that agrees with the existing prediction bias rather than one that provides more accurate supervision. The reported detection improvements suggest that the proposed strategy is useful, but they do not independently establish the reliability of the source-selection mechanism. A direct evaluation of the selected targets against ground-truth oriented bounding boxes would provide stronger evidence that ISA consistently chooses the more geometrically accurate candidate.

  • ISA performs one source selection after warm-up T=6T=6 and retains it for the remainder of training till epoch 1212. The target dimensions are recomputed as predicted orientation changes, but the source itself cannot change. This creates a mismatch: the detector evolves continuously, yet the method assumes the relative suitability of SAM and watershed is sufficiently stable after a single decision. For example, watershed may initially fit a predicted box better, but later training may reveal that SAM provides a more accurate extent. InstaPoint has no mechanism to correct that source assignment.

  • Point2RBox-v3's PGDM-Loss routes images to SAM or watershed according to image-level instance count. InstaPoint instead needs candidate extent targets for individual instances to compare their regression losses. This raises practical questions about mask-generation cost, preprocessing time, memory usage, and whether both candidate sources must be generated for every instance. The supplied evidence does not establish the complete computational cost of InstaPoint relative to the baseline.

  • The evaluation is conducted on only DOTA-v1.0 and DIOR-R, while Point2RBox-v3 report results on a broader range of datasets, including DOTA-v1.5, DOTA-v2.0, STAR, and RSAR. Evaluating InstaPoint on additional benchmarks, particularly the extended DOTA datasets, would provide stronger evidence of its generalizability. Alternatively, the authors could clarify why these benchmarks were not considered.

Questions for authors
  • Since IDC relies only on point spacing, how does it handle cases where objects share identical point distances but have vastly different sizes, aspect ratios, or background contrast?

  • Because ISA picks the mask that matches the model's current prediction, how do you prevent it from reinforcing confirmation bias? Can you provide a direct comparison of ISA-selected masks against Ground-Truth boxes to prove actual geometric accuracy?

  • Why freeze the supervision source permanently at epoch 66 while continuing to update orientation per step? Did you test continuous or soft-weighting mechanisms to prevent locking in noisy early decisions?

  • What are the measured preprocessing and training times, peak GPU memory, and overall computational costs relative to Point2RBox-v3? Must both candidate sources be generated and retained for every instance, or can the method avoid this cost in practice?

I am open to upgrading my score if the authors satisfactorily address these concerns.

6 comments
AuthorsOpenReview ↗

Response to Reviewer P9sk (1/3)

We thank the reviewer for the careful review and constructive feedback. We address the comments below.

W1–Q1. Objects with similar spacing but different characteristics

IDC does not directly assess instance-specific size, aspect ratio, or background contrast. It provides a prediction-independent spatial prior for initialization, which ISA subsequently refines by comparing the candidate extent targets with the detector’s prediction. The two candidate extent targets are the alternative width-height targets derived from the SAM and watershed masks.

The following DOTA-v1.0 assignment diagnostics use the candidate rotated boxes (RBoxes) and predicted orientations recorded when ISA fixes source assignments after six warm-up epochs. Object size is measured as wh\sqrt{wh}, where ww and hh are the width and height of the GT RBox at the training resolution. Selection accuracy measures how often the candidate with higher GT IoU is selected, while mean target IoU measures the average overlap of the selected candidates with the GT RBoxes. GT is used only for evaluation.

Here, sis_i is the distance to the nearest same-class point divided by the class-specific spacing reference; smaller values indicate closer same-class neighbors. We analyzed dense instances (si≤1s_i \leq 1), which account for 64.9% of DOTA instances. Under Eqs. (6)–(7), with ρ=1\rho=1, IDC assigns watershed to all these instances, although the relative quality of the two candidates differs across object-size groups.

Object sizeProportion (%)SAM IoU $ > $ WS IoU (%)IDC acc. (%)ISA acc. (%)
$ < 16$ pixels11.231.768.389.2
16–32 pixels60.139.760.378.8
32–64 pixels25.359.140.969.6

The table reports the proportion of dense instances in each size group. These groups cover approximately 96.6% of the dense set. “SAM IoU > WS IoU” denotes the percentage of instances where SAM has higher GT IoU than watershed (WS).

For dense 32–64-pixel instances, SAM is better in 59.1% of cases, so IDC’s watershed assignment is correct in only 40.9%. ISA improves selection accuracy in all three size groups, including an increase from 40.9% to 69.6% in the 32–64-pixel group. These results show that ISA refines the spatial initialization across object sizes within the same IDC density category. We have added size-wise results for 0.5<si≤10.5 < s_i \leq 1 to Table 11(a) of the revised manuscript.

To compare aspect ratio and background contrast among instances with similar normalized spacing, we restrict the analysis to instances with 0.5<si≤10.5 < s_i \leq 1, where IDC assigns watershed to all instances. Aspect ratio is measured as max⁡(w,h)/min⁡(w,h)\max(w,h)/\min(w,h) of the GT RBox. In the following tables, arrows denote IDC →\rightarrow ISA.

Aspect ratioInstancesSelection accuracy (%)Mean target IoU (%)
1–1.512,09442.4 → 72.662.9 → 71.1
1.5–212,18850.0 → 75.362.2 → 69.3
2–356,39056.2 → 75.768.0 → 72.7
3–527,45150.1 → 75.066.2 → 72.9
≥5\geq54,21941.4 → 77.150.1 → 60.9

To examine how ISA behaves across different levels of background contrast, we use CIELAB, a color space in which distances approximately reflect perceived color differences. We measure the color difference (ΔE\Delta E) between the mean colors inside the GT RBox and in a surrounding background ring. Larger values indicate stronger color contrast between these regions. We define low, medium, and high contrast using the one-third and two-thirds quantiles of the full diagnostic population's contrast distribution.

Background contrastInstancesSelection accuracy (%)Mean target IoU (%)
Low26,47349.9 → 77.359.5 → 66.8
Medium31,49649.6 → 75.363.9 → 70.9
High34,40650.0 → 73.569.0 → 75.2
Insufficient background19,96761.9 → 75.371.2 → 73.9

"Insufficient background" indicates that less than 20% of the surrounding ring remains after excluding pixels inside other GT boxes.

ISA improves both selection accuracy and mean target IoU across all aspect-ratio and contrast groups. For low-contrast instances, selection accuracy increases from 49.9% to 77.3%, and mean target IoU from 59.5% to 66.8%.

AuthorsOpenReview ↗

Response to Reviewer P9sk (2/3)

W2–Q2. Direct evaluation against ground-truth geometry

To directly evaluate whether ISA selects the more geometrically accurate extent target, we compare the candidate RBoxes against GT RBoxes using the same selection-time records and predicted orientations described above. Selection accuracy measures how often the selected candidate has higher GT IoU than the alternative.

Selection methodSelection accuracy (%)Mean target IoU (0–1)
IDC55.60.6238
ISA76.70.6804
GT-based selection of the better candidate—0.7052

ISA corrects 72.0% of initially incorrect IDC assignments, while changing 19.6% of initially correct assignments to the inferior candidate. Thus, individual errors remain, but ISA provides a net improvement in both selection accuracy and target geometry. The GT-based row is an evaluation-only upper bound. Neither IDC nor ISA uses GT RBoxes for supervision-source assignment or training. The submitted Fig. 2 provides complementary evidence with orientation fixed to the GT angle: mean target IoU increases from 0.638 to 0.671 on DOTA-v1.0 and from 0.615 to 0.651 on DIOR-R.

W3–Q3. Freezing the supervision source

After ISA selects the supervision source, the width-height target is recomputed at each iteration using the current predicted orientation. To examine whether updating the source itself would also help, we tested two repeated reassignment schedules. We also compared hard source selection with soft weighting, which combines both candidate losses after IDC warm-up.

Assignment settingE2E AP50\mathrm{AP}_{50}Two-stage (FCOS) AP50\mathrm{AP}_{50}
Epochs 7, 9, and 1160.9066.37
Every epoch from 7 to 1261.2067.01
Soft weighting (τ=0.05\tau=0.05)61.5467.34
Once, at epoch 762.5367.74

Neither repeated schedule improves over one-time reassignment. Under the every-epoch schedule, 17.2–23.2% of instances switch sources at each update, and 8.5–11.0% switch sources and then switch back at the next selection. These continued changes may reduce supervision consistency. With SAM caching, each additional selection epoch takes about 39 min longer than a fixed-source epoch. We therefore retain one-time reassignment, which achieves higher AP and avoids the extra computation required for repeated construction of both candidate extent targets.

Soft weighting is applied after six epochs of IDC warm-up and gives a larger weight to the candidate with the lower size loss. At each iteration, it weights the SAM size loss by αi=sigmoid((ℓiWatershed−ℓiSAM)/τ)\alpha_i=\mathrm{sigmoid}((\ell_i^{\mathrm{Watershed}}-\ell_i^{\mathrm{SAM}})/\tau) and the watershed size loss by 1−αi1-\alpha_i. The temperature τ=0.05\tau=0.05 controls how strongly the weighting favors the lower-loss candidate. The weights are recomputed at each iteration and treated as constants during backpropagation. One-time ISA also achieves higher AP than soft weighting in both protocols in this comparison.

W4–Q4. Preprocessing, training, memory, and inference costs

We measured computational costs under the same hardware setup (a single RTX 4090 with batch size 2). SAM caching means precomputing and reusing SAM masks during training.

MethodPreprocessingE2E training timePeak allocated GPU memory
Point2RBox-v3 (baseline) (reproduced)None10 h 19 min≈7.3 GB
InstaPoint, with SAM cachingIDC: ≤1\leq 1 min; SAM cache: 1 h 11 min18 h 39 min≈7.3 GB
InstaPoint, without cachingIDC: ≤1\leq 1 min27 h 22 min≈7.3 GB

The cache occupies 182 MB. Including cache generation, the cached pipeline takes approximately 19 h 50 min. Training alone takes 1.81 times as long as the baseline, mainly due to constructing per-instance extent targets. Peak allocated GPU memory, recorded in the training logs, is similar across the three settings (approximately 7.3 GB).

With SAM caching, we separately profiled the ISA selection and fixed-source phases:

Training phaseTime per iteration (s)Peak allocated GPU memory (MB)
ISA selection1.2796,693
Fixed-source phase0.9106,972

The time difference corresponds to about 39 additional minutes over 6,400 iterations in the single ISA selection epoch.

Both candidates are required when ISA compares supervision sources, but this comparison is performed only at the designated reassignment stage. Once the source is fixed, subsequent iterations recompute the selected extent target using the current predicted orientation without repeating the two-source comparison. At inference, ISA adds no modules or parameters to the detector.

Note that no mask generation or supervision-source assignment is performed at inference.

AuthorsOpenReview ↗

Response to Reviewer P9sk (3/3)

W5. Generalization to additional benchmarks

We extended the evaluation to DOTA-v2.0, STAR, and RSAR using the same IDC–ISA procedure without introducing new dataset-specific absolute distance thresholds.

DatasetMethodE2E AP50\mathrm{AP}_{50}Two-stage (FCOS) AP50\mathrm{AP}_{50}
DOTA-v2.0Point2RBox-v3 (baseline)34.4741.28
DOTA-v2.0InstaPoint (ours)35.1442.08
STARPoint2RBox-v3 (baseline)14.6019.60
STARInstaPoint (ours)15.5020.10
RSARPoint2RBox-v3 (baseline)40.8045.96
RSARInstaPoint (ours)42.1047.10

All values are AP50\mathrm{AP}_{50}, and Point2RBox-v3 results are taken from its paper. The baseline run in W4–Q4 was used for computational cost measurements, not to obtain the AP results in this table. InstaPoint improves over Point2RBox-v3 in both end-to-end and two-stage (FCOS) protocols across all three datasets, providing further evidence of generalization beyond DOTA-v1.0 and DIOR-R. We will release the corresponding configurations and evaluation submission files.

Due to an issue with the evaluation server, we have not yet received the DOTA-v1.5 results. We will provide them as soon as evaluation is completed.


We have incorporated the additional results and analyses presented above, along with the corresponding clarifications, into the revised manuscript and supplementary material.

Reviewer_hJBaOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes InstaPoint, an instance-adaptive supervision assignment framework for point-supervised oriented object detection. Building upon Point2RBox-v3, it introduces Instance Density Calibration (IDC) to initialize instance-wise supervision selection between SAM and watershed masks, followed by Instance-wise Supervision Assignment (ISA) to refine the selection using prediction-based regression losses. Experiments on DOTA-v1.0 and DIOR-R demonstrate consistent improvements under both end-to-end and two-stage detection protocols.

Critical strengths
  1. The paper identifies a well-motivated limitation of patch-wise supervision assignment and provides quantitative evidence supporting instance-level adaptation.

  2. The proposed framework improves supervision assignment without modifying the detector architecture or introducing additional training objectives.

  3. The method achieves consistent improvements on DOTA-v1.0 and DIOR-R under both end-to-end and two-stage detection protocols.

Critical weaknesses
  1. The proposed method largely extends Point2RBox-v3 by replacing patch-wise supervision selection with instance-wise heuristics, while retaining the original detector, supervision sources, and training objectives. Its methodological novelty beyond existing supervision assignment strategies is relatively limited.

  2. ISA assumes that the supervision source closest to the current detector prediction is more reliable, but prediction consistency does not necessarily imply geometric correctness. Inaccurate predictions may favor incorrect supervision and reinforce existing localization errors.

  3. IDC computes class-specific reference distances using nearest neighbors from all classes, while instance-level distances consider only same-class neighbors. This inconsistency may lead to unreliable density estimates in mixed-class clusters and undermine the proposed calibration mechanism.

  4. Although the paper provides module ablations, it lacks comparisons with simpler instance-wise supervision selection strategies. Consequently, it remains unclear whether the improvements originate from the specific IDC–ISA design or merely from changing the assignment granularity.

Questions for authors
  1. What are the key methodological contributions of InstaPoint beyond changing the supervision assignment granularity from patches to instances?

  2. How does ISA avoid reinforcing inaccurate detector predictions? Can the authors measure how often its selected supervision source is closer to the ground-truth object extent?

  3. Why is the class-specific reference distance computed using all-class nearest neighbors, while the instance distance uses only same-class neighbors? How does this affect mixed-class dense scenes?

  4. Can the authors compare IDC–ISA against simpler instance-wise selection strategies to establish the necessity of the proposed design?

4 comments
AuthorsOpenReview ↗

Response to Reviewer hJBa (1/2)

We thank the reviewer for the careful review and constructive feedback. We address the comments below.

W1–Q1. Methodological contribution

Point2RBox-v3 selects a source of extent supervision—SAM or watershed—for each patch by thresholding the number of instances it contains. All instances within the patch therefore share the same source.

This work addresses the problem of assigning a suitable supervision source to each instance based on candidate extent targets. The two candidate extent targets are the alternative width-height targets derived from the SAM and watershed masks. The relative suitability of these sources varies with the spatial configuration and visual characteristics of each instance. However, under point supervision, ground-truth object extent and orientation are unavailable, making it difficult to directly determine which candidate is more suitable. We propose an instance-adaptive supervision assignment framework that combines spatial initialization with one-time prediction-guided reassignment. Instance Density Calibration (IDC) initializes source assignments using class-calibrated local point spacing, and Instance-wise Supervision Assignment (ISA) refines them after warm-up by comparing candidate extent targets with the detector’s current prediction using the size loss. Our key contribution lies in designing a procedure for assessing the relative suitability of imperfect candidate extent targets and selectively using them as instance-specific training signals, going beyond simply using multiple supervision sources together.

We further test the ISA criterion by selecting the candidate with the larger size loss instead of the smaller one. On DOTA-v1.0, this reversed criterion yields 59.55/64.92 E2E/FCOS AP50\mathrm{AP}_{50}, compared with 62.53/67.74 for InstaPoint. This supports prediction–supervision consistency as a useful signal for source assignment.

W2–Q2. Reliability of ISA

IDC assigns sources independently of detector predictions during warm-up, so prediction-guided selection starts only after the detector has been trained for six warm-up epochs. ISA compares the two candidate extent targets with the detector’s prediction and revises 42.9% of assignments. Sources are then fixed, so later predictions do not feed back into source selection.

The following DOTA-v1.0 assignment diagnostics use the candidate rotated boxes (RBoxes) and predicted orientations recorded when ISA fixes source assignments after six warm-up epochs.

Selection accuracy measures how often the candidate with higher GT IoU is selected, while mean target IoU measures the average overlap of the selected candidates with the GT RBoxes. Prediction IoU instead measures the overlap between the detector’s prediction and the GT RBox at selection time. At selection, 91.8% of instances have a prediction IoU of at least 0.5. ISA’s selection accuracy is 72.4% for the 0.5–0.7 group and 79.2% for the ≥0.7\geq0.7 group. GT RBoxes are used only for diagnostic evaluation.

Selection methodSelection accuracyMean target IoU (0–1)
IDC55.6%0.6238
ISA76.7%0.6804

ISA corrects 72.0% of initially incorrect IDC assignments, while incorrectly changing 19.6% of initially correct assignments. The submitted Fig. 2 provides complementary evidence with orientation fixed to the GT angle: mean target IoU improves from 0.638 to 0.671 on DOTA-v1.0 and from 0.615 to 0.651 on DIOR-R. Thus, although individual errors remain, ISA substantially improves both overall selection accuracy and the geometric quality of the selected extent targets.

AuthorsOpenReview ↗

Response to Reviewer hJBa (2/2)

W3–Q3. All-class and same-class neighbor distances

The same-class definition of did_i yields a higher mean IoU for the instances on which the two definitions disagree, while changing the neighbor pool for ℓc\ell_c has only a small effect on the initial assignments. The two distances serve different roles: ℓc\ell_c provides a class-specific reference scale, whereas did_i measures local same-class separation. We examine these design choices by modifying the neighbor definitions in Eqs. (4) and (5) separately, keeping the rest of IDC unchanged. Both analyses use candidate RBoxes and predicted orientations recorded at ISA selection on DOTA-v1.0.

Replacing the same-class distance in Eq. (5).

We first replace the same-class nearest-neighbor distance did_i with an all-class nearest-neighbor distance, while keeping ℓc\ell_c fixed. Under Eq. (7) with ρ=1\rho=1, this changes the source assignment only when a neighbor of another class lies within ℓc\ell_c but no same-class neighbor does.

Neighbors within ℓc\ell_cSame class (ours)All-classSAM mean IoU (0–1)Watershed mean IoU (0–1)
Same-class neighbor(s)WatershedWatershed0.5790.631
Only other-class neighbor(s)SAMWatershed0.5030.477

When a same-class neighbor is nearby, both definitions select watershed, which performs better. When only other-class neighbors are nearby, the all-class distance would switch the assignment to watershed, whereas the original same-class did_i keeps SAM, which has a higher mean IoU. This supports the same-class definition used in Eq. (5).

Replacing the all-class reference in Eq. (4).

We then recompute ℓc\ell_c using only same-class neighbors, while keeping Eq. (5) and the assignment rule unchanged. Assignment agreement is the percentage of instances receiving the same source as under the original IDC rule.

Definition of ℓc\ell_cMean target IoU (%)Assignment agreement (%)
Same-class neighbors, class-wise62.2897.3
All-class neighbors, class-wise (ours)62.38100.0

Using only same-class neighbors to estimate ℓc\ell_c changes only 2.7% of IDC assignments and slightly reduces mean target IoU.

W4–Q4. Comparison with simpler instance-wise source assignment

We added a fixed-distance instance-wise rule that conservatively restricts SAM to well-separated instances. SAM is assigned only when the nearest and second-nearest same-class points are more than 192 and 256 pixels away, respectively, and watershed otherwise. The manually chosen 192-pixel threshold is approximately 4ℓglobal4\ell_{\mathrm{global}}, where ℓglobal=47.7\ell_{\mathrm{global}}=47.7 pixels is a spacing reference shared across classes. We obtain this reference by applying the same procedure used for ℓc\ell_c in Eqs. (1)–(4) to all-class nearest-neighbor distances of training points pooled across classes. The 256-pixel condition prevents SAM assignment when multiple same-class neighbors lie just beyond 192 pixels.

MethodE2E AP50\mathrm{AP}_{50}Two-stage (FCOS) AP50\mathrm{AP}_{50}
Point2RBox-v3 (baseline)59.6166.09
Fixed-distance instance-wise rule61.1267.02
InstaPoint62.5367.74

With the same candidate-generation and training settings, the fixed rule also outperforms always-SAM (60.89/66.62) and always-watershed (57.36/63.55). This supports the benefit of instance-wise source assignment. However, the fixed rule relies on manually chosen, dataset-specific pixel thresholds. InstaPoint automates this choice by estimating class-specific spacing references from the training point annotations and refining source assignments using detector predictions. In the same selection-time diagnostic, mean target IoU increases from 0.6163 with the fixed rule to 0.6238 with IDC and 0.6804 with ISA.

Ethics review flag

We noticed that the submission was flagged for ethics review, but the review does not describe a specific ethical concern. Could the reviewer clarify the issue underlying this flag so that we can address it directly and provide any necessary information?


We have incorporated the additional results and analyses presented above, along with the corresponding clarifications, into the revised manuscript and supplementary material.

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer hJBa,

Thank you again for your time and thoughtful review of our paper.

With about three days remaining in the discussion period, we wanted to briefly follow up on our responses and the revised manuscript. The key results relevant to your comments are summarized below:

  • Contribution and baselines (W1, W4): On DOTA-v1.0, InstaPoint achieves 62.53/67.74 AP50 (E2E/FCOS) versus 61.12/67.02 for the fixed-distance rule, supporting the benefit of IDC–ISA over a simpler instance-wise rule.
  • ISA reliability (W2): GT-based diagnostics show selection accuracy increasing from 55.6% to 76.7% and mean target IoU from 0.6238 to 0.6804, indicating improved target quality.
  • IDC neighbor definitions (W3): When only other-class neighbors lie within the IDC threshold, all-class local spacing would select watershed despite SAM’s higher mean IoU (0.503 vs. 0.477). Separately, using same-class neighbors to estimate the class-specific reference changes only 2.7% of IDC assignments.

We would appreciate your feedback on whether our responses and revisions address your concerns and would be happy to clarify any remaining points before the discussion closes.

Best regards,

The Authors

AuthorsOpenReview ↗

Final Note Before the Discussion Deadline

Dear Reviewer hJBa,

As the discussion period draws to a close in a few hours, we wanted to leave a brief final note. We hope our previous responses and summary have addressed your concerns. If any questions remain, please let us know, and we will respond promptly.

We would be grateful if you could take the clarifications, additional analyses, and revisions into account in your final assessment.

Thank you again for your time and thoughtful feedback.

Best regards,

The Authors

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your thoughtful comments and suggestions. Your feedback helped us clarify the roles of IDC and ISA and strengthen the experimental evaluation through direct diagnostics of supervision selection, comparisons with simpler assignment strategies, and further analyses of reassignment and training cost.

We hope the additional explanations and evidence help address your questions. Should any points need further clarification, we would be happy to continue the discussion during the remaining discussion period.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.