acceptodds
Under review as a conference paper at ICLR 2027

One Observation, One Identity: Joint Association and Truth Discovery for Remote-Sensing Agents

Abstract

Interpreting objects across optical, SAR, and temporal observations requires resolving two interdependent questions: which reports describe the same physical object, and which reported attribute values are correct. Incorrect associations can mix evidence from different objects and distort attribute estimates, while informative attributes can help resolve otherwise ambiguous matches. We introduce OneID-Agent, which assigns a single shared latent identity to all claims attached to the same physical observation, coupling object association with attribute truth inference. This formulation preserves the common observational origin of related claims and allows evidence from multiple attributes to inform a consistent association. Attribute beliefs jointly refine the shared identity, while the resulting association determines how each claim contributes to object-specific truth estimates. Physical constraints, contextual reliability, and provenance budgets constrain this feedback and keep inference grounded in the available observations. To support query answering, the agent integrates identity and attribute beliefs with evidence coverage and conflict checks to decide whether the available evidence supports an answer or further observations are needed. We evaluate agent execution on RSTIBench and joint inference against an independently annotated reference. Under matched inputs, shared identity improves assignment accuracy and attribute truth prediction compared with assigning separate identities to individual claims. Auxiliary reliability fitting further improves probability quality and answer coverage at the evaluated risk threshold.

Primary area:
datasets and benchmarks
Keywords:
Multimodal Remote Sensing, Object Association, Truth Discovery, Joint Inference.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_fWNKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper targets an important challenge, i.e., object matching and report aggregation, in remote-sensing analysis. By assigning a single shared identity to all claims derived from the same primitive observation, it enforces consistency among claims regarding the same entities/facts under observation. The authors have proposed a sound technical framework, achieving good results (accuracy). My main concern comes form the lack of clear information regarding how the influence of individual components/factors is assessed without consideration of others, especially the shared-identity mechanism and fine-tuning, which is critical in the claimed contribution. Additionally, the test set looks limited and it remains not crystal clear how the proposed approach compares with more established benchmark methods.

Critical strengths
  1. The paper addresses the challenging problem of aligning observations collected from different sensors, modalities, time periods, and processing pipelines. The research topic carries practical value in establishing a solid foundation for many downstream tasks that rely on accurate object matching to mitigate bias or corruption in follow-up attribute and change estimates.

  2. The main concept is intuitive and well justified. The approach has used semantic clues to reweight promising candidates than generating candidates that do not physically match.

  3. The strong reported pair precision reviews that the method performs effectively in cases where mistakes in object merge could incur substantial costs.

Critical weaknesses
  1. The agent-level comparison may interfere and negatively impact judgment on the effects of task-specific fine-tuning when the contribution to performance improvement may be wrongly attributed to other factors than the true cause or major influencer.

  2. Less details about the baseline methods have been provided, and therefore, the paper does not make it clear if the baselines ahve received the same treatment in terms of training, tools and runtime constraints while the primary identity evaluation relies on only 94 held-out examples.

  3. The compared methods are kind of limited. In particular, probabilistic record linkage, data association, and joint tracking approaches should be put in the scope.

Questions for authors

Q1. Why the final configuration wasn’t the one with the highest F1 score or optimal tuned parameters?

Q2. What was the mean runtime and the number of inference iterations per sample?

10 comments
AuthorsOpenReview ↗

Response to Reviewer fWNK (1/7):

Thank you for recognizing the value of our research question and shared-identity approach. Your observation that incorrect object matches affect subsequent attribute estimates and change judgments is one of the main reasons we developed OneID. We also note your concerns about the contributions of fine-tuning and the shared-identity module, the evaluation scale of 94 image pairs, and comparisons with existing methods. To address these concerns, we added agent experiments with fixed fine-tuned weights, five-fold cross-validation, and comparisons with existing record-linkage and joint-tracking methods.

We address your main comments and questions below and hope these responses resolve your concerns. In addition, the RSTI dataset, data from the supplementary experiments, and training-set JSON files are available via this link(https://drive.google.com/drive/folders/1yHFd4XA-RldwuJcEyLzblVHs2VpetcDr?usp=sharing). We plan to submit the revised manuscript before November 18 (UTC+08:00), together with the corresponding OneID source code. If you have any further questions, please feel free to reply, and we will promptly provide additional details about the paper. We welcome your questions and appreciate your attention!

AuthorsOpenReview ↗

Response to Reviewer fWNK (2/7): W1

Weakness 1. Do the agent-level gains conflate the contributions of task-specific fine-tuning and shared-identity inference?

Response: In Table 1 of the original paper, only OneID-Agent received SFT on RSTIBench. The general-purpose baselines used weights that had not been fine-tuned on this dataset. As you point out, that comparison did not isolate the gains from task-specific fine-tuning and the OneID module. We therefore added experiments that use the same fine-tuned weights with either basic detection tools or the OneID module. The settings and results are given below.

1. Why did we use this comparison? Our initial goal was to validate the SFT trajectories we constructed, enabling the agent to use tools for remote-sensing interpretation on RSTIBench and providing a working platform for the subsequent identity-inference experiments. The original paper therefore mainly reported the performance of the complete agent. We agree that establishing the contribution of OneID requires comparisons under the same fine-tuning conditions.

2. How were the additional experiments designed? We ran experiments with Qwen3-4B and Qwen2.5-7B on Task4 and Task5, the two tasks that use the OneID module. We evaluated (1) the base model answering directly, denoted by none; (2) the base and SFT models using basic detection tools, denoted by det; and (3) the base and SFT models using OneID, denoted by oneid. Table R1 reports the results. For each backbone, sft_det and sft_oneid use the same fine-tuned weights.

Table R1. Agent performance across training and tool configurations.

BranchMacroSeq.ToolRec.Norm.Arg.Ans.AttemptBlockedStepsRobustnessE2E
Qwen3-4B
4b_base_none0.000.000.000.000.02——1.00100.00—
4b_base_det38.119.7911.335.6622.2699.540.282.5199.9635.18
4b_sft_det41.137.2743.9235.7520.6493.882.505.3892.7438.52
4b_base_oneid70.619.6577.2034.8930.9263.4156.436.5187.2736.91
4b_sft_oneid89.9265.8776.2251.1636.1890.837.874.3895.8860.77
Qwen2.5-7B
7b_base_none12.150.000.000.000.000.00—4.0156.415.64
7b_base_det29.222.528.115.929.2287.216.543.7094.8627.14
7b_sft_det54.2922.9439.5838.6821.6794.031.433.5698.9943.97
7b_base_oneid56.6911.1967.5540.6615.7753.7354.797.1466.1730.34
7b_sft_oneid80.2830.9189.7960.5635.0079.6717.936.0793.1451.60

*Base = base weights; SFT = supervised fine-tuning. Steps is the mean number of executed actions per trajectory. All other metrics use a 0–100 scale.

3. How much does OneID improve final answers? In the Ans. column of Table R1, the score increases from 20.64 to 36.18 for 4b_sft_det → 4b_sft_oneid, and from 21.67 to 35.00 for 7b_sft_det → 7b_sft_oneid, gains of 15.54 and 13.33 points, respectively. These experiments show that adding OneID improves final-answer content scores even with the fine-tuned weights held fixed, supporting the effectiveness of the module.

4. How do we assess the contribution of shared identity itself? The agent experiments above measure the effect of adding the entire OneID module. To examine the shared-identity design, Table 2 of the original paper holds detections, the candidate graph, reports, and calibration conditions fixed, and replaces per-claim association U with per-observation shared identity J. TruthAcc increases from 71.87% to 73.56%, a gain of 1.69 percentage points. The agent experiments show how OneID improves final answers; the inference experiment further shows that assigning the multiple attributes of one observation to a shared object improves attribute-truth judgments.

AuthorsOpenReview ↗

Response to Reviewer fWNK (3/7): W2-1

Weakness 2. Does the main identity evaluation use only 94 image pairs, and are training, tools, and execution budgets comparable?

Response: Thank you for raising these questions about evaluation scale and comparability. We clarify two points. (1) Table R2 lists the ground-truth identity pairs and annotated object attributes. Although the number of image pairs is relatively limited, our study concerns individual objects, and a single image pair can contain dozens of objects. The number of image pairs alone therefore does not fully describe the scale of the evaluation; the object-level annotations also provide statistical support for assessing identity association and attribute judgments. (2) Beyond the original split of 374 image pairs for fitting and 94 for evaluation, we conducted five-fold cross-validation. Each of the 467 valid image pairs is evaluated once as a held-out sample, and we report 95% confidence intervals. We describe the data split, results, and comparison conditions below.

1. How was the dataset split? Task4 and Task5 originally contained 468 image pairs: 96 RGB–SAR pairs and 372 temporal optical pairs. We split them into 374 Train pairs and 94 Eval pairs to fit and evaluate the reliability and calibration parameters used in identity association and attribute-truth inference. This split is for the inference module and is separate from the split used for the agent's SFT trajectories. During a subsequent check, we found and removed 1 invalid sample containing no target objects. The additional experiments therefore use 467 image pairs, as detailed in Table R2. Our focus is the association of physical objects and the inference of their attributes, and the experimental data demonstrate OneID's effectiveness.

Table R2. RSTI Dataset Details.

Data subsetImage pairsGround-truth objectsGround-truth identity pairsTool detectionsTrue identity pairs among tool detectionsAnnotated object attributes
RGB–SAR951,6313232,293845,122
Temporal optical3725,6871,1016,3141,06525,233
Total46773181,4248,6071,14930,355

The table lists ground-truth objects and identity pairs in the images separately from the detections and corresponding pairs actually returned by the tools. This distinguishes the objects present in the imagery from the observations available for identity inference, keeping missed detections separate from identity-association errors.

2. How was five-fold cross-validation conducted? We used the following procedure:

(1) We stratified the 467 image pairs by RGB–SAR and temporal task into five folds of 94/94/93/93/93. Each image pair remains intact within one fold.

(2) For each fold, we refit source reliability and contextual calibration parameters using only the other four training folds. Hyperparameters are also selected within the training folds. The held-out fold is excluded from fitting and parameter selection.

(3) After fitting, we evaluate once on the held-out fold. We then pool the five folds into out-of-fold results covering all 467 image pairs and report 95% CIs.

Reviewer_JpcYOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents OneID-Agent, a remote-sensing agent framework for jointly resolving object identities and conflicting attribute reports across sensors and time. Its core method assigns a shared latent identity to all reports from each observation and alternates identity association and attribute estimation, incorporating physical constraints, calibrated tool reliability, and provenance-based weighting. Experiments on RSTIBench and an independently annotated image-pair subset report improvements in tool execution, association precision, and attribute estimation.

Critical strengths
  1. The observation-level identity formulation provides a coherent way to handle multiple reports from the same detection. Attribute evidence can revise an observation’s association while reallocating its other reports consistently, avoiding fragmented assignments within the proposed framework.

  2. The inference ablations hold observations, candidate graphs, and calibration access fixed. Shared identity improves attribute-truth accuracy from 71.87% to 73.56%, with a reported confidence interval excluding zero, providing evidence for this component’s contribution.

Critical weaknesses

1.Comparisons with established joint inference methods are limited. The association experiments primarily compare internal variants. Stronger comparisons with probabilistic entity-resolution or joint association-and-classification methods would help establish the contribution beyond combining existing modeling principles.

2.The precision–recall trade-off needs stronger characterization. The default configuration achieves 98.61% pair precision but only 58.20% recall, compared with 68.85% recall for hard matching. Although F1 improves, results at a single operating point do not establish how much of the precision gain comes from better association versus more conservative rejection. The 94-image-pair test set also limits evidence for generalization.

3.Important implementation and evaluation details cannot be verified from the provided manuscript. The cited appendices are absent, including details of calibration, candidate construction, inference settings, and answerability thresholds. This particularly limits assessment of the answering, further-acquisition, and abstention mechanism.

Questions for authors

1.Can the core inference method be compared with established joint association or probabilistic entity-resolution approaches using the same detections, candidate graph, reports, and calibration data?

2.Can the authors provide precision–recall curves and comparisons at matched precision or recall? Please also report the numbers of observations, correspondence pairs, and attribute facts represented by the 94 held-out image pairs.

3.Can the missing supplementary material be provided, particularly the calibration protocol, inference initialization and stopping criteria, and query-level results supporting answering, further acquisition, and abstention?

8 comments
AuthorsOpenReview ↗

Response to Reviewer JpcY (1/5):

Thank you for your encouraging assessment of OneID. We first clarify the data and settings for the shared-identity experiments. (a) Dataset. As stated in the paper, Task4 and Task5 originally contained 468 image pairs: 96 RGB–SAR pairs and 372 temporal optical pairs. We split these into 374 training pairs and 94 evaluation pairs to calibrate the reliability model and evaluate shared-identity inference. We estimate the calibration parameters on the fitting samples and then freeze them. On held-out samples, we alternate updates to the identity-association distribution AA and attribute-truth distribution QQ using the observed evidence, and evaluate the results. (b) Additional experiments. During a subsequent check, we identified and removed 1 invalid sample containing no target objects. The additional experiments therefore use 467 image pairs: 95 RGB–SAR pairs and 372 temporal optical pairs. Table R1 gives the details.

Table R1. RSTI dataset details (Task4+Task5).

Data subsetImage pairsGround-truth objectsGround-truth identity pairsTool detectionsTrue identity pairs among tool detectionsAnnotated object attributes
RGB–SAR951,6313232,293845,122
Temporal optical3725,6871,1016,3141,06525,233
Total46773181,4248,6071,14930,355

In addition, the RSTI dataset, data from the supplementary experiments, and training-set JSON files are available via this link(https://drive.google.com/drive/folders/1yHFd4XA-RldwuJcEyLzblVHs2VpetcDr?usp=sharing). We plan to submit the revised manuscript before November 18 (UTC+08:00), together with the corresponding OneID source code. If you have any further questions, please feel free to reply, and we will promptly provide additional details about the paper. We welcome your questions and appreciate your attention!

AuthorsOpenReview ↗

Response to Reviewer JpcY (2/5): Q1

Q1. Can OneID be compared with existing record-linkage or joint-association methods using the same inputs?

Response: We adapted MATCH by Guo et al. (2010) [1] and MM-STBF–JTC by Yang et al. (2014) [2] (abbreviated as JTC below) to the image-pair tasks, and compared them on Task4 and Task5 of RSTI. Table R2 reports results computed from the pooled out-of-fold predictions across all five folds.

Table R2. Operating points on pooled five-fold out-of-fold predictions from Task4 and Task5 (467 image pairs).

MethodPrecisionRecallF1PairBrier ↓IDAccIDECE ↓
F0.98340.62050.76090.18700.85690.1062
JθNP0.97080.63710.76930.19570.86180.0664
J+NP0.96970.61270.75090.19290.85400.0789
MATCH (0.5)0.51130.70760.59360.32650.71510.1644
MATCH (0.6)0.52220.67620.58930.32750.71940.1677
MATCH (0.7)0.53590.56570.55040.33550.71480.1663
MATCH (0.8)0.56650.37080.44820.33850.69900.1520
JTC (0.3)0.42570.63360.50930.50330.61910.3739
JTC (0.6)0.43280.63360.51430.50830.62740.3795
JTC (0.9)0.44790.57270.50270.51570.63110.3669

Values are on a 0–1 scale. Parentheses after MATCH and JTC give the decision threshold.

1. How does OneID compare with existing joint-association methods? At the operating points in Table R2, F achieves precision 0.9834, recall 0.6205, and F1 0.7609. MATCH at threshold 0.5 has higher recall (0.7076), with precision 0.5113 and F1 0.5936, reflecting a different precision–recall trade-off. The results show that our method achieves the best balance between precision and recall among the methods compared.

2. Why did we choose these two papers? They represent the two lines of work raised in your question: joint entity resolution and joint association–classification.

MATCH [1] by Guo et al. jointly handles record linkage and conflicting-value fusion through a k-partite graph and soft uniqueness constraints. MM-STBF–JTC [2] by Yang et al. jointly estimates target state and class in a random finite set framework. They serve as comparison methods for joint linkage and fusion, and joint association and classification, respectively. Existing remote-sensing detection and association methods tend to emphasize detection and matching, with less attention to object-level attribute truths across multiple observation sources. EvidentialRAG [3] and BayesRAG [4] also address conflicts across information sources or evidence fusion, but their focus on retrieval-augmented generation involves different kinds of data and different objectives from our task of remote-sensing identity association and attribute-truth inference.

OneID offers two advantages over these methods. A. Sharing identity and using attribute evidence to revise association. For RGB–SAR and temporal image pairs, we assign all attribute reports from the same observation to a shared identity. We construct candidates using location, time, and scale constraints, retain the option of no correspondence, and allow attribute evidence to revise identity association. B. Fusing conflicting evidence to support answers about specific objects. We use source reliability to fuse conflicting reports from different tools and limit repeated evidence from the same source. The resulting associations and attribute estimates support the agent's answers about specific objects, attributes, and times.

We would also welcome suggestions for more suitable comparison methods and would promptly run additional tests to improve the comparison.

AuthorsOpenReview ↗

Response to Reviewer JpcY (3/5): Q2

Q2. How much of the high precision comes from conservative rejection? Can you provide PR curves and the actual scale of the held-out samples?

Response: Table R3 shows that at the same recall, F reduces the false-positive count from 83 to 12 relative to J+N, increasing precision from 89.57% to 98.34%. F thus excludes incorrect associations more accurately while recovering the same number of true correspondences.

Table R3. Five-fold cross-validation results of VI variants on Task4 and Task5 (467 image pairs).

VariantpredTPFPP [95% CI]R [95% CI]F1 [95% CI]PairBrier↓TruthAccIDAccIDECE↓
H10843667180.3376 [0.3101, 0.3663]0.3185 [0.2922, 0.3460]0.3278 [0.3000, 0.3547]0.27990.77250.78000.1502
S6903613290.5232 [0.4859, 0.5602]0.3142 [0.2880, 0.3416]0.3926 [0.3654, 0.4209]0.26210.79040.79690.1344
U———————0.80270.81130.1226
J9967222740.7249 [0.6963, 0.7517]0.6284 [0.6000, 0.6558]0.6732 [0.6480, 0.6974]0.24060.81960.83780.1035
J+8857221630.8158 [0.7889, 0.8400]0.6284 [0.6000, 0.6558]0.7099 [0.6873, 0.7325]0.22010.82420.84520.0928
J+N796713830.8957 [0.8726, 0.9151]0.6205 [0.5921, 0.6482]0.7332 [0.7157, 0.7506]0.20530.82700.84980.0857
J+NP726704220.9697 [0.9545, 0.9799]0.6127 [0.5842, 0.6405]0.7509 [0.7413, 0.7605]0.19290.82880.85400.0789
Jθ8547511030.8794 [0.8558, 0.8996]0.6536 [0.6256, 0.6806]0.7499 [0.7329, 0.7688]0.20910.82640.85110.0825
JθNP754732220.9708 [0.9562, 0.9807]0.6371 [0.6089, 0.6644]0.7693 [0.7588, 0.7777]0.19570.82950.86180.0664
F−P736694420.9429 [0.9238, 0.9575]0.6040 [0.5754, 0.6319]0.7363 [0.7236, 0.7491]0.21390.82590.84700.1236
F725713120.9834 [0.9713, 0.9905]0.6205 [0.5921, 0.6482]0.7609 [0.7535, 0.7673]0.18700.83110.85690.1062

1. How much does precision improve at the same recall? J+N and F both recover 713 true correspondences, giving recall 62.05%. F makes 71 fewer false pairings while recovering the same number of true pairs, demonstrating that the precision gain can come from more effective rejection of incorrect associations.

2. Can fewer predictions recover more true correspondences? Compared with F−P, F makes 11 fewer predictions but recovers 19 more correct pairs and produces 30 fewer false pairs. This shows that F can revise association decisions to improve both error control and the recovery of true correspondences.

3. PR curves and comparisons

The two PR plots below show Task4's 95 RGB–SAR pairs and Task5's 372 temporal optical pairs separately. Each embedded table reports the operating points for that task. Pair precision, recall, and F1 in Tables R2 and R3 are calculated from pooled counts of the out-of-fold predictions for both tasks across all five folds.

Figure R1. Pairwise PR curves for Task4 (optical–SAR). https://imgur.com/a/WaoGVsm

Figure R2. Pairwise PR curves for Task5 (temporal optical images). https://imgur.com/a/PzVrcp6

Reviewer_qALKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes OneID-Agent, a remote-sensing agent that jointly infers object identities and attribute truths from optical, SAR, and temporal observations. To do this, this method alternates observation-object association update and true attribute value update. The framework also checks evidence sufficiency to answer, acquire additional evidence, or abstain. The authors introduce RSTIBench for tool-execution evaluation, with separately adjudicated identity and attribute-truth annotations for inference evaluation.

Critical strengths

A meaningful approach to coupling identity association and attribute-truth inference in remote-sensing agents. Sharing one identity across an observation’s claims allows attribute evidence to revise associations while keeping those claims together. The controlled inference-level ablations provide evidence that observation-level sharing improves attribute-truth estimation over independent claim association, supporting the value of this approach.

Critical weaknesses

The experiments do not clearly establish the framework’s contribution to agent-level performance. OneID-Agent is fine-tuned on RSTIBench, but comparable training for the Table 1 baselines is not clearly documented. Consequently, the reported gains may partly reflect task-specific training rather than shared-identity inference. The paper also lacks comparison with an existing tool-augmented remote-sensing agent such as OpenEarthAgent, which it cites. Although Table 2 provides useful inference-level ablations, it does not establish the corresponding improvement in final agent answers. Tables 12–13 assess submission decisions rather than answer-content correctness and therefore do not resolve this gap. Comparisons under matched training conditions, including an ablation that changes only the inference module within the same fine-tuned agent, are needed to substantiate the system-level contribution.

Questions for authors

  1. Controlled comparisons. Were the Table 1 baselines fine-tuned on the same RSTIBench trajectories? Could the authors compare against OpenEarthAgent with an aligned tool interface and comparable training, and evaluate the same fine-tuned agent with and without shared-identity inference using final answer accuracy and coverage?
  2. Workflow and acquisition loop. The policy, tool execution, evidence-state rules, and answer/acquisition decisions are distributed across the main text and appendix. Could the authors provide a consolidated workflow and evidence showing whether executing the acquisition loop improves final answers over stopping after the initial inference?
  3. Relationship to prior work. Please distinguish the methodological contribution relative to data integration methods for joint entity resolution and truth discovery from the contribution relative to remote-sensing correspondence and agent methods. Which modeling principles are inherited, and what technical extensions and additional capabilities does OneID introduce?
  4. Benchmark scope. Which requirements of shared-identity evaluation are missing from the OpenEarthAgent benchmark and motivate RSTIBench? An evaluation on compatible tasks from that benchmark would help assess whether the benefits extend beyond the authors’ own dataset.
  5. Evaluation definition. Appendix B.6 describes E2E as a unified end-to-end score but does not specify its calculation. Could the authors provide the formula and clarify whether it directly measures final answer correctness?
12 comments
AuthorsOpenReview ↗

Response to Reviewer qALK (1/7): Q1-123

Thank you for your positive assessment of our work on shared identity. We also note your presentation score of 1 and your concern that Figure 2 does not clearly show how the agent proceeds from tool outputs to a final answer.

We would like to clarify this workflow. (1) Consider your example of RGB–SAR interpretation: after identifying a cargo vessel in an optical image, how does the agent determine that it is the same vessel as a particular target in the SAR image? (2) Our agent first calls detection tools to locate targets in both views, then uses their positions and attributes to determine which observations refer to the same vessel. It checks whether these observations provide enough evidence to answer the question. (3) We are revising the main text to walk through this vessel example alongside Figure 2, so readers can follow the tools, identity decisions, and answer for the same observation. Thank you for taking the time to examine our work.

In addition, the RSTI dataset, data from the supplementary experiments, and training-set JSON files are available via this link(https://drive.google.com/drive/folders/1yHFd4XA-RldwuJcEyLzblVHs2VpetcDr?usp=sharing). We plan to submit the revised manuscript before November 18 (UTC+08:00), together with the corresponding OneID source code. If you have any further questions, please feel free to reply, and we will promptly provide additional details about the paper. We welcome your questions and appreciate your attention!

$ $

Q1. Were the methods in Table 1 trained under the same conditions? How do final answers and coverage change when OneID is added to the same fine-tuned agent? How does it compare with OpenEarthAgent?

Response: In Table 1 of the paper, only OneID-Agent received task-specific fine-tuning. The general-purpose baselines used base weights without SFT.

1. Why did we use this comparison? Our initial goal was to evaluate the effectiveness of the chain-of-thought trajectories we constructed for SFT. The aim was to show that SFT enabled the agent to run experiments on RSTI and provided a working platform for subsequent identity-inference experiments. The original manuscript therefore did not separate the contributions of SFT and the shared-identity module.

2. How much does OneID improve final answers? We ran additional experiments with Qwen3-4B and Qwen2.5-7B on Task4 and Task5, the two tasks that use OneID. We evaluated (1) the base model answering directly, denoted by none; (2) the base and SFT models using basic detection tools, denoted by det; and (3) the base and SFT models using OneID, denoted by oneid. Table R1 reports the results. For each backbone, sft_det and sft_oneid use the same fine-tuned weights.

3. What do the results show? In the Ans. column of Table R1, the score increases from 20.64 to 36.18 for 4b_sft_det → 4b_sft_oneid, and from 21.67 to 35.00 for 7b_sft_det → 7b_sft_oneid. These results support the effectiveness of OneID and its role in helping the model identify and associate objects.

Table R1. Agent performance across training and tool configurations.

BranchMacroSeq.ToolRec.Norm.Arg.Ans.AttemptBlockedStepsRobustnessE2E
Qwen3-4B
4b_base_none0.000.000.000.000.02——1.00100.00—
4b_base_det38.119.7911.335.6622.2699.540.282.5199.9635.18
4b_sft_det41.137.2743.9235.7520.6493.882.505.3892.7438.52
4b_base_oneid70.619.6577.2034.8930.9263.4156.436.5187.2736.91
4b_sft_oneid89.9265.8776.2251.1636.1890.837.874.3895.8860.77
Qwen2.5-7B
7b_base_none12.150.000.000.000.000.00—4.0156.415.64
7b_base_det29.222.528.115.929.2287.216.543.7094.8627.14
7b_sft_det54.2922.9439.5838.6821.6794.031.433.5698.9943.97
7b_base_oneid56.6911.1967.5540.6615.7753.7354.797.1466.1730.34
7b_sft_oneid80.2830.9189.7960.5635.0079.6717.936.0793.1451.60

All metrics except Steps use a 0–100 scale. A dash indicates that the data cannot be calculated.

AuthorsOpenReview ↗

Response to Reviewer qALK (2/7): Q1-4

Q1. Were the methods in Table 1 trained under the same conditions? How do final answers and coverage change when OneID is added to the same fine-tuned agent? How does it compare with OpenEarthAgent?

4. How does OneID compare with OpenEarthAgent? We agree that comparisons with related systems are important. OneID-Agent and OpenEarthAgent overlap in some of their goals, but their different designs lead to differences in tool protocols, experimental settings, and evaluation objectives. Given these differences, weaker performance by OpenEarthAgent on our dataset was to be expected.

To examine this issue, we retained OpenEarthAgent's released weights and adapted the parts of its tool interface that differed from ours. We then evaluated it on RSTIBench. Because OpenEarthAgent was not fine-tuned on the same RSTIBench trajectories, Table R2 primarily measures its transfer to our tasks. It is not a controlled comparison under identical training conditions.

Table R2. Comparison with OpenEarthAgent.

BranchInst.ToolArg1ArgstrictArg2AnswerMacroSeq.ToolRec.Norm.Arg.AttemptBlockedStepsE2E
OpenEarth65.0359.7456.8234.0634.0964.8110.313.387.225.3828.06N/A4.6717.18
Qwen3-4B97.6371.7159.6327.2027.2059.2539.467.218.561.6284.833.281.7428.88
OneID99.6779.7864.4646.6946.7178.9078.1842.7258.0631.9483.7415.444.2844.90

Under these settings, OpenEarthAgent trails even Qwen3-4B on many metrics, with an E2E score of 17.18. The adapted OpenEarthAgent results should be interpreted with care: its original tool interface cannot be used directly with ours, and differences in protocols and training conditions limit comparability.

AuthorsOpenReview ↗

Response to Reviewer qALK (3/7): Q2-1

Q2. What is the complete workflow from tool execution to the final answer? Does acquiring additional evidence improve final answers?

Response: The workflow is shown in Figure 2 of the paper. Using paired RGB–SAR images as an example, it proceeds as follows:

  1. Planning. The agent develops a plan and selects tools based on the user's request for interpreting the paired RGB–SAR images.
  2. Detection. The agent first selects ObjectDetection_worker for the RGB image and DenoDetV2_worker for the SAR image. It calls these tools, converts their detections to bounding-box coordinates [x1, y1, x2, y2], and obtains target attributes.
  3. Identity and attribute inference. Using the detected boxes, the agent calls Qwen3-VL to identify attributes such as color, subtype, and size. OneID-Agent extracts and combines this information as input to the shared-identity module, which returns association states, such as Sar-only, RGB-Only, and Match, together with their confidence scores.
  4. Evidence-based decisions. The agent combines these observations into an evidence state and uses it to decide whether to obtain more evidence, answer the question, or end the interaction.

1. How is additional evidence acquired? The current agent cannot autonomously acquire additional evidence or detect further targets; this still requires a follow-up question from a user or a trigger from an external process. We have therefore not evaluated its ability to choose when to run additional detection or which tool to use. This is a limitation of the current system. To address your question, we added an experiment in which an external process triggers targeted additional detection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their thoughtful feedback. We appreciate their recognition of the practical importance of reliable object matching (fWNK), our coherent observation-level identity formulation (qALK, JpcY), and the controlled evidence for its contribution to attribute-truth estimation (qALK, JpcY).

Our work makes three connected contributions:

  • OneID-Agent for object-level remote-sensing interpretation. The framework selects and invokes tools for a user query, then uses OneID identities to organize detections across sensors and acquisition times into object-level records for answering.
  • OneID for joint identity association and truth discovery. A variational model gives each observation and its attached reports one shared latent identity. Spatial and temporal constraints define plausible matches. Calibrated report reliability depends on observation context, while provenance budgets limit the combined influence of related reports. Attribute evidence can revise an observation's identity and reallocate all its reports together.
  • RSTIBench for agent execution and object-level inference. Reference tool trajectories, intermediate evidence, and answers support agent evaluation. Independently adjudicated correspondences and attribute values on optical-SAR and temporal image pairs enable direct evaluation of matching and attribute inference.

During the rebuttal period, we provided the following additional experiments and clarifications:

  1. Better final answers after adding OneID to the same fine-tuned model (qALK, JpcY, fWNK). On Task4 and Task5, switching from basic detection tools to the full OneID module raises the final-answer content score (Ans., on a 0-100 scale) from 20.64 to 36.18 for Qwen3-4B and 21.67 to 35.00 for Qwen2.5-7B. Within each backbone, both conditions use the same SFT weights. These agent results complement the original inference ablation, where changing per-claim association U to shared identity J raises truth accuracy from 71.87% to 73.56% with inputs held fixed.
  2. Broader evaluation through five-fold cross-validation (JpcY, fWNK). Each of 467 valid image pairs is evaluated once out of fold; reliability fitting and hyperparameter selection use only the training folds. We report confidence intervals and dataset counts, including 8,607 tool detections and 30,355 annotated object attributes. Pooled out-of-fold results give F 98.34% precision, 62.05% recall, and 76.09% F1.
  3. Comparisons with established methods (JpcY, fWNK, qALK). We added image-pair adaptations of MATCH (Guo et al., 2010) and MM-STBF-JTC (Yang et al., 2014). At thresholds 0.5 for MATCH and 0.6 for JTC, their F1 scores are 59.36% and 51.43%, compared with 76.09% for F. We also evaluated an interface-adapted OpenEarthAgent as a transfer reference and clarified how OneID extends joint linkage and fusion to spatially and temporally scoped remote-sensing observations.
  4. Precision-recall analysis and configuration choice (JpcY, fWNK). We supplied task-specific PR curves. At the same 62.05% recall, J+N and F both recover 713 true pairs, while false pairings decrease from 83 to 12. This comparison shows improved error control while retaining the same number of true correspondences. We explain F's emphasis on avoiding false merges and report JθNP's advantages in F1 and identity calibration.
  5. Workflow, metrics, and execution details (qALK, JpcY, fWNK). We clarified the path from tool outputs to object records and answers, calibration and test-time freezing, agent stopping rules, and the distinction between final-answer content (Ans.) and the composite E2E score. In the reported Qwen3-4B runtime test, adding OneID reduces mean agent execution time from 18.38 to 11.69 seconds and executed steps from 5.38 to 4.38.

We will incorporate these results and clarifications into the revised manuscript and provide the complete supplementary material, with a clearer statement of the contributions and evaluation protocols.

Thank you for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.