- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper targets an important challenge, i.e., object matching and report aggregation, in remote-sensing analysis. By assigning a single shared identity to all claims derived from the same primitive observation, it enforces consistency among claims regarding the same entities/facts under observation. The authors have proposed a sound technical framework, achieving good results (accuracy). My main concern comes form the lack of clear information regarding how the influence of individual components/factors is assessed without consideration of others, especially the shared-identity mechanism and fine-tuning, which is critical in the claimed contribution. Additionally, the test set looks limited and it remains not crystal clear how the proposed approach compares with more established benchmark methods.
Critical strengths
-
The paper addresses the challenging problem of aligning observations collected from different sensors, modalities, time periods, and processing pipelines. The research topic carries practical value in establishing a solid foundation for many downstream tasks that rely on accurate object matching to mitigate bias or corruption in follow-up attribute and change estimates.
-
The main concept is intuitive and well justified. The approach has used semantic clues to reweight promising candidates than generating candidates that do not physically match.
-
The strong reported pair precision reviews that the method performs effectively in cases where mistakes in object merge could incur substantial costs.
Critical weaknesses
-
The agent-level comparison may interfere and negatively impact judgment on the effects of task-specific fine-tuning when the contribution to performance improvement may be wrongly attributed to other factors than the true cause or major influencer.
-
Less details about the baseline methods have been provided, and therefore, the paper does not make it clear if the baselines ahve received the same treatment in terms of training, tools and runtime constraints while the primary identity evaluation relies on only 94 held-out examples.
-
The compared methods are kind of limited. In particular, probabilistic record linkage, data association, and joint tracking approaches should be put in the scope.
Questions for authors
Q1. Why the final configuration wasn’t the one with the highest F1 score or optimal tuned parameters?
Q2. What was the mean runtime and the number of inference iterations per sample?
10 comments
Response to Reviewer fWNK (2/7): W1
Weakness 1. Do the agent-level gains conflate the contributions of task-specific fine-tuning and shared-identity inference?
Response: In Table 1 of the original paper, only OneID-Agent received SFT on RSTIBench. The general-purpose baselines used weights that had not been fine-tuned on this dataset. As you point out, that comparison did not isolate the gains from task-specific fine-tuning and the OneID module. We therefore added experiments that use the same fine-tuned weights with either basic detection tools or the OneID module. The settings and results are given below.
1. Why did we use this comparison? Our initial goal was to validate the SFT trajectories we constructed, enabling the agent to use tools for remote-sensing interpretation on RSTIBench and providing a working platform for the subsequent identity-inference experiments. The original paper therefore mainly reported the performance of the complete agent. We agree that establishing the contribution of OneID requires comparisons under the same fine-tuning conditions.
2. How were the additional experiments designed? We ran experiments with Qwen3-4B and Qwen2.5-7B on Task4 and Task5, the two tasks that use the OneID module. We evaluated (1) the base model answering directly, denoted by none; (2) the base and SFT models using basic detection tools, denoted by det; and (3) the base and SFT models using OneID, denoted by oneid. Table R1 reports the results. For each backbone, sft_det and sft_oneid use the same fine-tuned weights.
Table R1. Agent performance across training and tool configurations.
| Branch | Macro | Seq. | ToolRec. | Norm.Arg. | Ans. | Attempt | Blocked | Steps | Robustness | E2E |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-4B | ||||||||||
4b_base_none | 0.00 | 0.00 | 0.00 | 0.00 | 0.02 | — | — | 1.00 | 100.00 | — |
4b_base_det | 38.11 | 9.79 | 11.33 | 5.66 | 22.26 | 99.54 | 0.28 | 2.51 | 99.96 | 35.18 |
4b_sft_det | 41.13 | 7.27 | 43.92 | 35.75 | 20.64 | 93.88 | 2.50 | 5.38 | 92.74 | 38.52 |
4b_base_oneid | 70.61 | 9.65 | 77.20 | 34.89 | 30.92 | 63.41 | 56.43 | 6.51 | 87.27 | 36.91 |
4b_sft_oneid | 89.92 | 65.87 | 76.22 | 51.16 | 36.18 | 90.83 | 7.87 | 4.38 | 95.88 | 60.77 |
| Qwen2.5-7B | ||||||||||
7b_base_none | 12.15 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | — | 4.01 | 56.41 | 5.64 |
7b_base_det | 29.22 | 2.52 | 8.11 | 5.92 | 9.22 | 87.21 | 6.54 | 3.70 | 94.86 | 27.14 |
7b_sft_det | 54.29 | 22.94 | 39.58 | 38.68 | 21.67 | 94.03 | 1.43 | 3.56 | 98.99 | 43.97 |
7b_base_oneid | 56.69 | 11.19 | 67.55 | 40.66 | 15.77 | 53.73 | 54.79 | 7.14 | 66.17 | 30.34 |
7b_sft_oneid | 80.28 | 30.91 | 89.79 | 60.56 | 35.00 | 79.67 | 17.93 | 6.07 | 93.14 | 51.60 |
*Base = base weights; SFT = supervised fine-tuning. Steps is the mean number of executed actions per trajectory. All other metrics use a 0–100 scale.
3. How much does OneID improve final answers? In the Ans. column of Table R1, the score increases from 20.64 to 36.18 for 4b_sft_det → 4b_sft_oneid, and from 21.67 to 35.00 for 7b_sft_det → 7b_sft_oneid, gains of 15.54 and 13.33 points, respectively. These experiments show that adding OneID improves final-answer content scores even with the fine-tuned weights held fixed, supporting the effectiveness of the module.
4. How do we assess the contribution of shared identity itself? The agent experiments above measure the effect of adding the entire OneID module. To examine the shared-identity design, Table 2 of the original paper holds detections, the candidate graph, reports, and calibration conditions fixed, and replaces per-claim association U with per-observation shared identity J. TruthAcc increases from 71.87% to 73.56%, a gain of 1.69 percentage points. The agent experiments show how OneID improves final answers; the inference experiment further shows that assigning the multiple attributes of one observation to a shared object improves attribute-truth judgments.
Response to Reviewer fWNK (3/7): W2-1
Weakness 2. Does the main identity evaluation use only 94 image pairs, and are training, tools, and execution budgets comparable?
Response: Thank you for raising these questions about evaluation scale and comparability. We clarify two points. (1) Table R2 lists the ground-truth identity pairs and annotated object attributes. Although the number of image pairs is relatively limited, our study concerns individual objects, and a single image pair can contain dozens of objects. The number of image pairs alone therefore does not fully describe the scale of the evaluation; the object-level annotations also provide statistical support for assessing identity association and attribute judgments. (2) Beyond the original split of 374 image pairs for fitting and 94 for evaluation, we conducted five-fold cross-validation. Each of the 467 valid image pairs is evaluated once as a held-out sample, and we report 95% confidence intervals. We describe the data split, results, and comparison conditions below.
1. How was the dataset split? Task4 and Task5 originally contained 468 image pairs: 96 RGB–SAR pairs and 372 temporal optical pairs. We split them into 374 Train pairs and 94 Eval pairs to fit and evaluate the reliability and calibration parameters used in identity association and attribute-truth inference. This split is for the inference module and is separate from the split used for the agent's SFT trajectories. During a subsequent check, we found and removed 1 invalid sample containing no target objects. The additional experiments therefore use 467 image pairs, as detailed in Table R2. Our focus is the association of physical objects and the inference of their attributes, and the experimental data demonstrate OneID's effectiveness.
Table R2. RSTI Dataset Details.
| Data subset | Image pairs | Ground-truth objects | Ground-truth identity pairs | Tool detections | True identity pairs among tool detections | Annotated object attributes |
|---|---|---|---|---|---|---|
| RGB–SAR | 95 | 1,631 | 323 | 2,293 | 84 | 5,122 |
| Temporal optical | 372 | 5,687 | 1,101 | 6,314 | 1,065 | 25,233 |
| Total | 467 | 7318 | 1,424 | 8,607 | 1,149 | 30,355 |
The table lists ground-truth objects and identity pairs in the images separately from the detections and corresponding pairs actually returned by the tools. This distinguishes the objects present in the imagery from the observations available for identity inference, keeping missed detections separate from identity-association errors.
2. How was five-fold cross-validation conducted? We used the following procedure:
(1) We stratified the 467 image pairs by RGB–SAR and temporal task into five folds of 94/94/93/93/93. Each image pair remains intact within one fold.
(2) For each fold, we refit source reliability and contextual calibration parameters using only the other four training folds. Hyperparameters are also selected within the training folds. The held-out fold is excluded from fitting and parameter selection.
(3) After fitting, we evaluate once on the held-out fold. We then pool the five folds into out-of-fold results covering all 467 image pairs and report 95% CIs.
Response to Reviewer fWNK (1/7):
Thank you for recognizing the value of our research question and shared-identity approach. Your observation that incorrect object matches affect subsequent attribute estimates and change judgments is one of the main reasons we developed OneID. We also note your concerns about the contributions of fine-tuning and the shared-identity module, the evaluation scale of 94 image pairs, and comparisons with existing methods. To address these concerns, we added agent experiments with fixed fine-tuned weights, five-fold cross-validation, and comparisons with existing record-linkage and joint-tracking methods.
We address your main comments and questions below and hope these responses resolve your concerns. In addition, the RSTI dataset, data from the supplementary experiments, and training-set JSON files are available via this link(https://drive.google.com/drive/folders/1yHFd4XA-RldwuJcEyLzblVHs2VpetcDr?usp=sharing). We plan to submit the revised manuscript before November 18 (UTC+08:00), together with the corresponding OneID source code. If you have any further questions, please feel free to reply, and we will promptly provide additional details about the paper. We welcome your questions and appreciate your attention!