acceptodds
Under review as a conference paper at ICLR 2027

TRACE: Towards Interpretable Carbon Emission Estimation via Reasoning-to-Prediction from Satellite Imagery

Abstract

Accurate monitoring of urban carbon emissions is essential for climate mitigation, yet satellite-based deep models remain difficult to interpret: they predict an emission value without revealing which scene properties support it. Vision-Language Models (VLMs) have the potential to bridge this semantic gap; however, applying them directly to carbon-emission estimation remains challenging because they are prone to visual hallucinations and lack mechanisms tailored to continuous regression. We propose TRACE (Textual Reasoning-Augmented Carbon Estimation), a textual reasoning-augmented framework compatible with different visual encoders in which an Analyst VLM generates image-grounded environmental analyses and a downstream Predictor maps visual and textual features to an emission estimate. In the absence of ground-truth analytical annotations, teacher-generated pseudo-analyses are used to initialize the Predictor, followed by alternating Analyst–Predictor optimization through Periodical Evolution. Experiments using data from five cities show improvements over image-only baselines across four visual encoders. TRACE achieves relative R² gains of 3.6%–6.8%, while blind human evaluation shows improved factual grounding and carbon relevance over unfine-tuned VLM analyses.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Visual Reasoning, Reinforcement Learning, Remote Sensing, Carbon Emission Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ckWWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRACE, a framework that uses VLM-generated textual analyses as an intermediate representation for 1 km monthly carbon-emission regression from satellite imagery. TRACE has two training phases. In cold-start initialization, a Qwen2.5-VL-72B Teacher receives the image, city, month, and the ODIAC-derived emission value z, and generates pseudo-analyses used to initialize an MLP Predictor that fuses image and text embeddings. In Periodical Evolution, a smaller Analyst VLM is trained with GRPO using two rewards: a Judge-based Analysis Score for grounding and relevance, and a Prediction Reward measuring whether the generated analysis reduces the frozen Predictor’s relative error. The Analyst and Predictor are updated alternately. Experiments cover five cities and four visual backbones, with R²/NRMSE/MAPE improvements, reward and text-input ablations, Judge scores, and a 50-case blind human audit.

Critical strengths
  1. The paper treats inspectable textual analysis as an intermediate variable for continuous carbon regression rather than using a VLM directly as a regressor. The reported gains across ViT, RemoteCLIP, GeoRSLIP, and DINOv3-sat suggest that the text branch is not tied to a single visual representation. This is a meaningful contribution for remote-sensing carbon estimation and for interpretable VLM applications.
  2. Periodical Evolution addresses two coupled problems: the absence of expert-written analysis labels and the risk that a fixed Predictor can be exploited by the Analyst. The design—Teacher pseudo-analysis initialization, dual-reward RL, and alternating Predictor retraining—is coherent. The reward ablation in Table 4 supports the claim that Analysis-only and Prediction-only training are each insufficient and that the combination is better.
Critical weaknesses
  1. The most serious concern is that cold-start pseudo-analyses are generated by a Teacher conditioned on the reference emission value z. The prompt explicitly places z in the context and permits quantitative or contextual justification. Pseudo-analyses may therefore contain emission values, intensity descriptions, or other label-correlated cues. The Predictor may learn to rely on label-like text rather than visual evidence, which would undermine both the interpretability claim and the validity of the Prediction Reward. The authors do not quantify label leakage in pseudo-analyses, nor do they provide no-label, randomized-label, or numeric-filtered cold-start controls.
  2. The statistical rigor and external validity are insufficient. The main results appear to use a single tile-level split without multi-seed means/variance or significance tests. Although tiles are disjoint, random tile splits can still leave spatial autocorrelation leakage across neighboring tiles. All five cities are relatively developed urban areas, and New York remains difficult (TRACE-4B R² ≈ 0.1397 in Table A1), indicating weak absolute performance in hard settings. The human audit uses only 50 cases, two evaluators, and only the 4B pair; inter-rater agreement and Judge-human correlation are not reported.
Questions for authors
  1. How is cold-start label leakage ruled out? Please quantify the fraction of Teacher pseudo-analyses that contain the reference emission value, numeric ranges, or label-recoverable cues. I recommend adding three cold-start controls: (i) Teacher without z, (ii) Teacher with randomized z, and (iii) pseudo-analyses with numeric/unit expressions filtered out. Also report a text-only Predictor and sentence-deletion/replacement tests on the generated analysis. If the Predictor relies mainly on label-like text in pseudo-analyses, the visual-attribution claim needs to be substantially revised.
  2. Are the composite reward and GRPO training robust? The paper defines Analysis Reward and Prediction Reward but does not clearly specify the composite weighting, normalization, sensitivity to n z, and the batch-mean rule Rbatch. Please report multi-seed variance, reward-weight ablations, and ablations of the Tau threshold and batch-mean rule, including how batch composition affects the reward. Also clarify the number of Periodical Evolution stages, stopping criteria, KL coefficient, and clipping threshold. 3.Judge bias and human-audit statistical power. The Teacher and Judge are both Qwen2.5-VL-72B, so self-preference or style bias is possible. Please report the correlation between Judge scores and human scores, inter-rater agreement between the two evaluators, confidence intervals for the 50-case audit, and a human audit for the 2B model or a larger audit sample. It would also be useful to test whether the Analysis Reward favors longer, more specific, but not necessarily more faithful analyses.
  3. Spatial split, label uncertainty, and baseline fairness. Please clarify whether the tile-level split is random-tile or spatial-block, and whether neighboring-tile leakage was checked. ODIAC 1 km monthly labels have uncertainty, and MAPE is unstable at low emission values; a label-noise/uncertainty analysis would strengthen the evaluation. For baselines, please add direct VLM regression, caption+fusion pipelines, remote-sensing VLMs such as GeoChat/RS-LLaVA, and a same-size Analyst without RL, so that the gain can be attributed to the method rather than to model scale or an extra text branch.
  4. Inference cost and faithfulness of the “inspectable” analysis. TRACE requires running the Analyst VLM at inference, followed by text encoding and the Predictor. Please report latency, memory/FLOPs, and failure rates relative to image-only inference. For interpretability, please add faithfulness tests: delete or replace key visual-evidence sentences in the analysis, ask humans to predict emissions from the analysis alone.
8 comments

We thank the reviewer for the thorough review and concrete experimental suggestions. Below, we quantify label-related content in cold-start texts, report 5-seed results (mean ±\pm std), detail the composite reward and GRPO settings, add bootstrap CIs and cross-family evaluators to the audit, clarify the spatial split, include the requested baselines, and report inference costs and sentence-deletion faithfulness tests.

[W1 / Q1] Implementation clarifications. (i) In Appendix A.1, "quantitative" was a typo for "qualitative". (ii) The Teacher generates pseudo-analyses on the Predictor-training subset exclusively to initialize the cold-start Predictor; Teacher texts on the Analyst-training subset are used only for cold-start early stopping. The Analyst is never trained on pseudo-analyses and never observes ground-truth labels; subsequent Predictors are trained solely on the Analyst's own outputs, and test inference uses no pseudo-analyses.

Label content in generated texts (excluding prompt boilerplate such as "1 km" and "2022"):

Text SourcennAny NumberNumber + Emission UnitWithin ±0.5%\pm 0.5\% of Raw Ref.
Teacher pseudo-analyses19,2251.25%0.05%0.05%
TRACE-4B (test)4,94057.4%0.00%0.12%
Qwen3-VL-4B (test)4,94037.8%0.00%0.14%

Filtering number+emission-unit tokens would alter only 0.05%0.05\% of cold-start texts (1.25%1.25\% if stripping all numbers), and TRACE-4B matches the raw ODIAC reference within ±0.5%\pm 0.5\% (without requiring a unit match) no more often than Qwen3-VL-4B (0.12%0.12\% vs. 0.14%0.14\%). Downstream, the RL-trained Analyst consistently outperforms the same-size Qwen3-VL Analyst across all four visual encoders (see [W2 / Q2]).

Text-input controls:

Predictor InputR2R^2vs. TRACE-2B
Text only (no image)0.5210.521−0.247-0.247
Image + empty text0.709±0.0120.709 \pm 0.012−0.058-0.058
Image + shuffled analyses0.744±0.0130.744 \pm 0.013−0.023-0.023
Image + TRACE-2B analyses0.768±0.0030.768 \pm 0.003—
  • Text only: Without images, R2R^2 drops to 0.5210.521 (vs. 0.7160.716 image-only), confirming the text is not a surrogate label.
  • Replacement test: Pairing each tile with another tile's analysis from the same city and split (one permutation per seed) preserves domain style and city context while breaking tile correspondence. R2R^2 drops across all seeds (−0.023-0.023), proving reliance on tile-specific content. The remaining gain over empty text reflects retained city-level priors, comparable to an explicit city/month one-hot baseline (0.7520.752).
  • Sentence deletion: Detailed in [Q5]; confirms reliance on both visual-evidence and contextual sentences.

[W2 / Q2] Five-seed evaluation (R2R^2, mean ±\pm std):

EncoderImage-onlyQwen3-VL-4BTRACE-4Bvs. Image-onlyvs. Qwen3-VL-4B
ViT-L/160.716±0.0110.716 \pm 0.0110.748±0.0100.748 \pm 0.0100.773±0.0060.773 \pm 0.006+0.057+0.057+0.025+0.025
DINOv3-sat0.842±0.0040.842 \pm 0.0040.845±0.0090.845 \pm 0.0090.862±0.0020.862 \pm 0.002+0.020+0.020+0.017+0.017
RemoteCLIP0.749±0.0090.749 \pm 0.0090.766±0.0130.766 \pm 0.0130.785±0.0020.785 \pm 0.002+0.036+0.036+0.020+0.020
GeoRSCLIP0.783±0.0150.783 \pm 0.0150.799±0.0060.799 \pm 0.0060.814±0.0060.814 \pm 0.006+0.031+0.031+0.015+0.015

All gains are positive on every seed, with TRACE-4B's std no larger than any baseline's.

Composite reward:

R=0.4⋅AS⋅1[Rpred>0]+0.6⋅Rpred,Rpred={tanh⁡(2Δ)if e∅≥τ=0.05RˉbatchotherwiseR = 0.4 \cdot \text{AS} \cdot \mathbb{1}[R_{\text{pred}} > 0] + 0.6 \cdot R_{\text{pred}}, \quad R_{\text{pred}} = \begin{cases} \tanh(2\Delta) & \text{if } e_{\emptyset} \ge \tau = 0.05 \\ \bar{R}_{\text{batch}} & \text{otherwise} \end{cases}

AS∈[0,1]\text{AS} \in [0,1] contributes only when Rpred>0R_{\text{pred}} > 0; both terms are bounded, requiring no further normalization. Δ=e∅−efused\Delta = e_{\emptyset} - e_{\text{fused}}, where e∅e_{\emptyset} is the relative error of the exact same fused Predictor given an empty string (clarifying Eq. (2)). For tiles with e∅<5%e_{\emptyset} < 5\% (where relative improvement is ill-conditioned), Rˉbatch\bar{R}_{\text{batch}} assigns the neutral mean RpredR_{\text{pred}} of non-fallback responses in the same rollout step (512512 prompts).

GRPO & Periodical Evolution (EasyR1): Group size =5= 5; low-variance KL β=0.01\beta = 0.01; clip =0.2/0.3= 0.2 / 0.3; AdamW (lr=1×10−6\text{lr} = 1\times 10^{-6}, weight decay 0.010.01); temp =1.0= 1.0, top-p=0.95p = 0.95; max length =1024= 1024 tokens. Two 1-epoch stages (≈37\approx 37 steps/stage), refitting the Predictor after each stage. Stopping criterion: validation composite reward rises from 0.4850.485 to 0.5160.516 in Stage 1 and plateaus at 0.517–0.5210.517\text{--}0.521 in Stage 2, so training stops after Stage 2. We evaluate the final Stage-2 Analyst and Predictor. Final checkpoints lie within 0.0020.002 of peak validation reward; KL stays low/stable, entropy stays within 0.88–0.920.88\text{--}0.92, and refitting the Predictor causes no reward drop (0.516→0.5190.516 \to 0.519). Curves are in the Appendix.

Reward ablation: Qwen3-VL-2B (0.7470.747), prediction-only (0.7510.751), analysis-only (0.7580.758), full composite (0.7680.768, +0.016+0.016 over pred-only, +0.009+0.009 over analysis-only).

[Q3] Self-preference audit. We added two cross-family evaluators (gpt-5.6-sol, gpt-6-astra) on the 5050 audit pair-cases, scoring factuality, carbon relevance, and hallucination-freeness on randomized, anonymized pairs given the image, city, and month:

EvaluatorTRACE / Tie / Qwen3-VL-4BMean Diff (1–10 scale)95% CI
Human 138 / 3 / 9+0.56+0.56[+0.30,+0.81][+0.30, +0.81]
Human 227 / 7 / 16+0.36+0.36[+0.06,+0.67][+0.06, +0.67]
Human audit34 / 5 / 11+0.52+0.52[+0.33,+0.71][+0.33, +0.71]
gpt-5.6-sol28 / 5 / 17+0.30+0.30[+0.03,+0.57][+0.03, +0.57]
gpt-6-astra30 / 2 / 18+0.31+0.31[+0.01,+0.61][+0.01, +0.61]
Qwen2.5-VL-72B Judge (0–1 scale)31 / 11 / 8+0.068+0.068[+0.032,+0.106][+0.032, +0.106]

Inter-rater agreement. The human reference uses Table 2's case-level audit scores. Strict-preference agreement is the fraction of matching directions, excluding cases with a tie from either evaluator. We use directions to avoid score-calibration differences (Human 2 scores 1.51.5 pts higher on average): Human 1 vs. 2 is 27/4127/41 (65.9%65.9\%); 72B Judge vs. human audit is 25/3625/36 (69.4%69.4\%); gpt-5.6-sol and gpt-6-astra vs. human audit are 29/4129/41 (70.7%70.7\%) and 31/4331/43 (72.1%72.1\%); and agreement between the two GPTs is 37/4337/43 (86.0%86.0\%). All evaluators consistently prefer TRACE-4B.

[Q4] Spatial split. Within each city, tiles are randomly partitioned 4:4:1:14:4:1:1 into Predictor-train, Analyst-train, validation, and test splits, keeping all months of a tile in the same split. Neighboring tiles may fall into different splits, yielding Moran's II (annual mean, 4-neighbor weights) of 0.79–0.990.79\text{--}0.99 in four cities and 0.190.19 in New York. This involves no label leakage: no test images or labels are used during training, and RpredR_{\text{pred}} is computed on tiles unseen by the Predictor. This evaluates spatial completion across unseen cells within mapped cities (Section 4.1) on an identical split across all methods.

Difficult cities (ViT, 5 seeds). Gains do not rely on high spatial autocorrelation: in New York (I=0.19I = 0.19), image-only reaches 0.0770.077, Qwen3-VL-4B 0.1070.107, and TRACE-4B 0.1190.119 (+0.042+0.042 over image-only across 5/55/5 seeds; low absolute R2R^2 reflects heavy-tailed labels with CV=4.4\text{CV} = 4.4). TRACE-4B also beats image-only on 5/55/5 seeds in Beijing (+0.095+0.095), Chicago (+0.090+0.090), Shanghai (+0.052+0.052), and Singapore (+0.068+0.068).

Label uncertainty & MAPE. Because ODIAC is a modeled proxy and MAPE is sensitive to near-zero denominators, we prioritize R2R^2 and set τ=0.05\tau = 0.05. Appendix statistics show right-skewed per-city labels (skewness 4.4–24.94.4\text{--}24.9; CV\text{CV} 0.8–4.40.8\text{--}4.4): outliers with ∣si∣>3|s_i| > 3 comprise 0.2–1.3%0.2\text{--}1.3\% of source monthly cells but have 5–100×5\text{--}100\times higher mean emissions. Thus, MAPE reflects low-emission cells while R2R^2 captures high-emission cells, motivating joint reporting of R2R^2, NRMSE, and MAPE.

Reviewer_JwNVOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents Textual Reasoning-Augmented Carbon Estimation (TRACE) framework which predicts the carbon emission from satellite images. They exploited pretrained encoders to generate the image and text embeddings and pass to a predictor to get the carbon emission output as predictions. They teacher model is used to generate pseudo-analyses which act as predictor initialization. The analyst is trained using GRPO with analysis reward and prediction reward. The analyst and predictors are trained in turns to get the carbon estimation. Experimental study uses 2022 planet imagery at 3m and ODIAC emission labels for 5 cities.

Critical strengths

A) It is interesting to see the carbon emission estimation from optical satellite imagery.

B) TRACE combines inspectable environmental analyses with features from visual encoders and trains a separate Predictor for continuous estimation.

C) A teacher VLM acts as generator to provide pseudo-labels for predictor initialization.

D) The analyst is optimized using two separate reward functions and the predictor is updated based on newly generated analyses using the regression objectives.

Critical weaknesses

A) I am not clear about the dataset mapping. The ODAIC provides carbon emissions at 1KM resolution where as the satellite imagery are from Planet at 3m per pixel. How the authors aligned the data at same image resolution. It is better to provide more clear details about data cleaning and aligning so that it is clear about image resolution. Also, it is better to have different year training and different year for testing splits.

B) The Analyst VLM and predictor are trained disjoint. It is better to see the learning curves.

C) The paper is poorly written with repetitive. Analyst and Predictor are not clear from the figures.

D) It is not clear that a VLM model how/what looks into the images and estimates carbon emissions.

E) It is not clear the framework is specific to carbon emission. Also authors claim “Jointly optimizing the Analyst VLM and predictor can result in moving-target instability” but there is not ablation to support this claim.

F) It is better to know that method account whether pseudo-analyses contain numbers or level words, and it doesn't test a Teacher without the label.

G) It is better to see a small test so that it tell us whether judge scores and human scores are aligned.

Questions for authors

Please see my pointed weakness. A clear answer to these might change my decision.

4 comments

We thank the reviewer for the detailed and constructive comments. We address each point below and have updated the manuscript accordingly.

[A] Spatial alignment. Using the native ODIAC grid (nominally 1 km1\text{ km}, or 30 arc-seconds30\text{ arc-seconds}) as our sampling unit, we crop co-registered 3 m3\text{ m} Planet mosaics directly to each cell's geographic footprint, yielding one RGB tile per cell per month (the cell's ground footprint, and hence the tile's pixel dimensions, varies with latitude). Each tile's label is the cell's monthly ODIAC emission value (converted from tonnes C to kt C\text{kt C}). Crucially, the emission inventory is never spatially resampled: high-resolution imagery is cropped to the native label grid.

Data cleaning. For each city and month, we select a Planet scene with <10%< 10\% cloud cover, apply the provider's cloud mask, and manually inspect RGB composites to discard tiles with residual clouds, heavy haze, or no-data pixels, as well as tiles with missing or non-positive ODIAC values. This yields 48,31448,314 valid monthly image–label pairs across 4,5754,575 unique spatial tiles (Beijing: 11,47511,475; Chicago: 8,7418,741; New York: 12,31212,312; Shanghai: 9,5159,515; Singapore: 6,2716,271).

Dataset split & scope. We partition unique spatial tiles 4:4:1:14:4:1:1 into Predictor-training, Analyst-training, validation, and test subsets, keeping all monthly observations of a given tile strictly within the same split. Our dataset covers 2022 (the year with complete co-registered 3 m3\text{ m} Planet imagery across all five cities); Section 4.1 clarifies that our evaluation focuses on spatial generalization across unseen tiles within these five cities in 2022.

[B] We added step-by-step validation curves across both RL stages (tracking composite reward, gated analysis score RA=AS⋅1[Rpred>0]R_A=\mathrm{AS}\cdot\mathbb{1}[R_{\mathrm{pred}}>0], and Prediction Reward) to the Appendix. For TRACE-4B:

  • Stage 1 (3737 steps): Validation composite reward rises steadily from 0.4850.485 to 0.5160.516, RAR_A from 0.6880.688 to 0.7510.751, and Prediction Reward from 0.3510.351 to 0.3600.360.
  • Stage 2 (3737 steps, Predictor retrained on Stage-1 analyses): Composite reward stabilizes at 0.517–0.5210.517\text{--}0.521 and RAR_A at 0.750–0.7590.750\text{--}0.759, with low, stable KL divergence throughout and no reward drop after refitting the Predictor (0.516→0.5190.516 \to 0.519). Given this plateau, we treat training as converged and stop after two stages.

[C] Module roles and interfaces (now explicitly labeled in Figures 1 and 2 captions, with subset and freezing schedules consolidated in Section 3):

  • Analyst = the VLM (Qwen3-VL-2B/4B). Input: satellite image, city, month. Output: qualitative textual analysis (never outputs numerical emission estimates).
  • Predictor = a lightweight MLP regressor. Input: concatenated image embedding and text embedding of the Analyst's output. Output: continuous emission prediction.

[D] Rather than outputting emission numbers, the VLM (Analyst) describes visible spatial features (building density, roads, industry, vegetation, water) and macro-context (city, season), which the Predictor fuses with visual features. To verify that the Predictor relies on tile-specific visual details in the text, we ran a controlled ablation:

Shuffled-analysis control (ViT, 2B Analyst, 5 seeds): Pairing each tile with another randomly chosen tile's analysis from the same city and split preserves domain vocabulary, style, and city context while breaking tile correspondence. Test R2R^2 drops from 0.7680.768 to 0.7440.744 (−0.023-0.023 consistently across all 5 seeds), whereas empty text yields 0.7090.709.

Why within-city shuffled text outperforms empty text. Because shuffling occurs within each city, every tile still receives a TRACE analysis from its own city discussing city-level context (Table 3), retaining city priors and comparable to an explicit city/month one-hot baseline (0.7520.752). Conversely, off-the-shelf RS-VLMs (EarthDial, GeoChat; R2R^2 of 0.7090.709, 0.7110.711) receive the exact same city/month prompt inputs but rarely mention the city in their outputs (6%6\% and 12%12\% vs. 4%4\% for generic short captions), carrying negligible macro-context and performing at the image-only level (0.7160.716). Thus, city/month context provides a strong baseline, while matching task-specific analyses to each tile further improves prediction. Table 3 shows TRACE-2B accurately grounding red-roofed housing, narrow streets, vegetation, and water (where Qwen3-VL-2B hallucinates commercial structures), while our failure-case appendix analyzes a Singapore scene where circular landscape features are misread as roundabouts.

[E] Domain specificity. While the high-level paradigm (Analyst generating text →\to regressor scoring utility →\to alternating stage-wise updates) is conceptually general, adapting it to a new domain requires a task-specific Predictor, domain prompts, and Judge rubrics; all components and evaluations here are tailored specifically to carbon emission estimation.

Moving-target rationale. Because GRPO computes relative advantages within a group of candidate analyses, updating the Predictor mid-stage would assign shifting rewards to the same text across steps. We revised the manuscript to state: "We keep the Predictor fixed within each Analyst stage so that the reward model remains stationary during policy optimization." Empirically, Stage 2 trains smoothly against the refitted Predictor without transition degradation.

[F] First, we corrected a typo in Appendix A.1 ("quantitative" →\to "qualitative"). Regarding numerical leakage and qualitative level words:

  1. Cold-start confinement. Teacher pseudo-analyses are used exclusively to initialize the cold-start Predictor and then discarded. The Analyst is never trained on pseudo-analyses and never sees ground-truth labels; subsequent Predictors are trained solely on the Analyst's own outputs.
  2. Negligible numerical content. Auditing generated texts (excluding prompt boilerplate like "1 km" and "2022") confirms label values are virtually absent:
Text SourcennAny NumberNumber + Emission UnitWithin ±0.5%\pm 0.5\% of Raw Ref.
Teacher pseudo-analyses19,2251.25%0.05%0.05%
TRACE-4B (test)4,94057.4%0.00%0.12%
Qwen3-VL-4B (test)4,94037.8%0.00%0.14%

Only 0.05%0.05\% of Teacher texts contain a number near the true label, and TRACE-4B falls within ±0.5%\pm 0.5\% of the raw ODIAC reference (unit match not required) no more often than Qwen3-VL-4B (0.12%0.12\% vs. 0.14%0.14\%). 3. Downstream evidence. The RL-trained Analyst consistently outperforms the same-size Qwen3-VL model across all four visual encoders (e.g., +0.025+0.025 R2R^2 on ViT and +0.017+0.017 on DINOv3 across 5 seeds). 4. Qualitative level words are intentional. Cold-start pseudo-analyses include qualitative intensity descriptors (e.g., high/low emission likelihood) to teach the initial Predictor to interpret qualitative judgments, so the Analyst is rewarded during RL when it correctly infers emission levels from visual and contextual evidence. Because the Analyst never sees labels, inaccurate assessments directly penalize its Prediction Reward.

[G] Across 5050 audited cases, two human annotators, our Qwen2.5-VL-72B Judge, and two cross-family LLMs (gpt-5.6-sol, gpt-6-astra) independently evaluated TRACE-4B vs. Qwen3-VL-4B analyses (humans and GPT models on randomized, anonymized pairs; the Judge by scoring each analysis separately):

EvaluatorTRACE-4B PreferredTieQwen3-VL-4B Preferred
Human 1 / Human 238 / 273 / 79 / 16
Qwen2.5-VL-72B Judge31118
gpt-5.6-sol / gpt-6-astra28 / 305 / 217 / 18

The human reference uses Table 2's case-level audit scores. Strict-preference agreement is the fraction of matching directions, excluding cases with a tie from either evaluator. We use directions to avoid score-calibration differences (Human 2 scores 1.51.5 pts higher on average): Human 1 vs. 2 is 27/4127/41 (65.9%65.9\%); 72B Judge vs. human audit is 25/3625/36 (69.4%69.4\%); gpt-5.6-sol and gpt-6-astra vs. human audit are 29/4129/41 (70.7%70.7\%) and 31/4331/43 (72.1%72.1\%); and agreement between the two GPTs is 37/4337/43 (86.0%86.0\%). All five evaluators consistently prefer TRACE-4B (added to the Appendix).

Thank you for the response. It answers some of my concerns, but many are still not satisfactory.

A) The reply lists features like building density and roads. But these show where activity may be, not how much fuel is burned. I am still not sure which image features are really linked to emissions, and the paper does not show this directly.

B) Error propagation: Wrong claims from the Analyst go into the Predictor retraining with no filter. What safeguards are there, and how do you check that errors do not build up from stage to stage?

Therefore, I want to keep my score.

Thank you for the follow up. We address both questions directly below:

(A) Linking observable image features to emissions. Our task is to estimate emissions from observable remote sensing evidence. If fuel consumption were directly known at the 1 km1\text{ km} scale, emissions would be computed by accounting rather than estimated. Indeed, the reference ODIAC inventory itself disaggregates national fuel consumption totals onto the 1 km1\text{ km} grid using observable spatial proxies (nighttime lights and power plant locations). Visible indicators of activity location and intensity (such as building density, road networks, and industrial infrastructure) are therefore the exact physical signals linked to this target. Empirically, our shuffled control within each city, where each tile retains identical city context, vocabulary, and style but receives another tile's analysis, consistently degrades test R2R^2 by 0.0230.023 across all 55 seeds (0.768→0.7440.768 \to 0.744). This directly confirms that visual descriptions specific to each tile carry genuine predictive signal.

(B) Safeguards and verification against error propagation. No estimation model is free of errors; the key questions are whether erroneous claims are rewarded and whether errors accumulate across stages:

  • Safeguards: Erroneous claims are prevented from being reinforced through two mechanisms: (1) the Judge's Analysis Score directly penalizes factual errors and hallucinations during RL, and (2) at each stage, the Predictor is retrained against ground truth emission labels, ensuring that inaccurate or unpredictive content receives no Prediction Reward.

  • Verification across stages: We explicitly monitor held out validation performance across the stage transition. Refitting the Predictor on Stage 1 analyses causes no degradation in validation composite reward (0.516→0.5190.516 \to 0.519), while the analysis score remains stable from 0.7500.750 to 0.7590.759 throughout Stage 2. Overall, TRACE consistently outperforms every baseline across all four visual encoders and five random seeds.

Reviewer_YPYKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TRACE, a multimodal framework for carbon emission estimation from remote sensing images. It uses a vision-language model (VLM) to generate analysis texts and combines them with image features for continuous value prediction, offering a new approach to interpretable remote sensing analysis.

Critical strengths

Monitoring urban carbon emissions is important for climate change mitigation, and the effort to explain model predictions offers a way to better understand the factors associated with emission levels. The paper is clearly written and well organized.

The dataset includes five cities and establishes links between satellite imagery and carbon emission values. It could serve as a useful resource for future studies on remote sensing-based carbon emission estimation.

Critical weaknesses

All data are from five cities in 2022, and the split tests unseen tiles within each city. This supports within-city spatial generalization, but not transfer to a new city, year, or season. Since the target is an ODIAC inventory proxy rather than direct measurements, the paper should be careful not to imply broad real-world monitoring validity from the current split alone.

The paper gives the individual reward definitions and a GRPO objective, but does not state the exact composite-reward weighting/normalization, several key optimization settings, or multi-seed variability and confidence intervals. The Analysis Reward is produced by a Qwen2.5-VL-72B Judge from the same family as the cold-start Teacher, while the independent audit covers only 50 cases and reports no inter-rater agreement. These choices leave open judge overfitting, label-informed cold-start effects, and training instability as explanations for part of the reported gains.

TRACE mainly follows a common SFT + GRPO training pipeline. The novelty appears limited.

Questions for authors

Can you provide the exact composite reward equation, including the Analysis/Prediction weights and normalization, as well as the values of n, GRPO group size, clipping/KL coefficients, stage lengths, and any other settings needed to reproduce Periodical Evolution. Please add reward/KL curves and results over multiple random seeds or confidence intervals.

Can you report a leave-one-city-out experiment, or at least a transfer test to a held-out city or year? If this is infeasible, please narrow the claims explicitly to unseen-tile generalization within the five observed cities.

How sensitive are the results to the Qwen2.5-VL-72B Judge? A cross-family Judge, a rubric with independently scored criteria, and inter-rater agreement for the human audit would help distinguish genuine grounding from evaluator coupling.

The cold-start Teacher receives the exact emission label, can you compare against a label-free Teacher or mask/perturb the numeric value and test whether the Predictor relies on label-correlated phrases? A shuffled-analysis or style-only control would further test reliance on grounded content rather than stereotyped wording.

Please clarify whether a structured metadata plus image baseline (city one-hot and month encoding, optionally simple geospatial features) or an UrbanVLP/attribute-scoring style feature baseline changes the incremental value attributed to RL-shaped analyses.

4 comments

We thank the reviewer for the thorough and constructive review. Below, we clarify our evaluation scope, detail the composite reward and training configurations, report 5-seed evaluations (mean ±\pm std), add a cross-family audit, and include the requested metadata, shuffled-text, and feature baselines, alongside two implementation clarifications.

[W1 / Q2] Our dataset covers five cities in 2022 using a within-city held-out tile split against the ODIAC inventory proxy. Given cross-city variations, Section 4.1 clarifies that our claims specifically address spatial generalization across unseen tiles within the five observed cities in a single year. Because all compared variants share the exact same split and visual features, differing strictly in text inputs, relative comparisons remain strictly controlled.

[W2 / Q1] Composite reward:

R=0.4⋅AS⋅1[Rpred>0]+0.6⋅Rpred,Rpred={tanh⁡(2Δ)if e∅≥0.05RˉbatchotherwiseR = 0.4 \cdot \text{AS} \cdot \mathbb{1}[R_{\text{pred}} > 0] + 0.6 \cdot R_{\text{pred}}, \quad R_{\text{pred}} = \begin{cases} \tanh(2\Delta) & \text{if } e_{\emptyset} \ge 0.05 \\ \bar{R}_{\text{batch}} & \text{otherwise} \end{cases}

AS∈[0,1]\text{AS} \in [0,1] is gated by 1[Rpred>0]\mathbb{1}[R_{\text{pred}} > 0] to contribute only when Rpred>0R_{\text{pred}} > 0; both terms are bounded, requiring no extra normalization. Rˉbatch\bar{R}_{\text{batch}} is the mean RpredR_{\text{pred}} across non-fallback responses in the same rollout step. Clarifying Eq. (2): e∅e_{\emptyset} is computed by feeding an empty text string to the exact same fused Predictor (not a separate image-only branch), so Δ=e∅−efused\Delta = e_{\emptyset} - e_{\text{fused}} strictly isolates the marginal contribution of the generated text.

GRPO & Predictor configurations. GRPO (EasyR1): Group size =5= 5; low-variance KL β=0.01\beta = 0.01; clip =0.2/0.3= 0.2 / 0.3; AdamW (lr=1×10−6\text{lr} = 1\times 10^{-6}, weight decay 0.010.01); rollout batch =512= 512 prompts; temp =1.0= 1.0, top-p=0.95p = 0.95; max length =1024= 1024 tokens. Training runs for two 1-epoch stages (≈37\approx 37 steps/stage), updating the Predictor after each stage. Both Judge and Teacher use Qwen2.5-VL-72B-Instruct. Predictor: Concatenates (⊕\oplus) the visual embedding with the 1024-d Qwen3-Embedding-0.6B text embedding into an MLP (ReLU, dropout 0.10.1). After each Analyst stage, the Predictor is fine-tuned from the previous stage using Adam (lr=1×10−3\text{lr} = 1\times 10^{-3}, cosine schedule, batch size 128128, up to 100100 epochs, early stopping patience 1010 on the validation set) and kept frozen during Analyst RL. Only cold-start early stopping uses the Analyst-training subset; final evaluations use the final Stage-2 models without checkpoint cherry-picking.

Five-seed evaluation (R2R^2, mean ±\pm std):

EncoderImage-onlyQwen3-VL-4BTRACE-4Bvs. Image-onlyvs. Qwen3-VL-4B
ViT-L/160.716±0.0110.716 \pm 0.0110.748±0.0100.748 \pm 0.0100.773±0.0060.773 \pm 0.006+0.057+0.057+0.025+0.025
DINOv3-sat0.842±0.0040.842 \pm 0.0040.845±0.0090.845 \pm 0.0090.862±0.0020.862 \pm 0.002+0.020+0.020+0.017+0.017
RemoteCLIP0.749±0.0090.749 \pm 0.0090.766±0.0130.766 \pm 0.0130.785±0.0020.785 \pm 0.002+0.036+0.036+0.020+0.020
GeoRSCLIP0.783±0.0150.783 \pm 0.0150.799±0.0060.799 \pm 0.0060.814±0.0060.814 \pm 0.006+0.031+0.031+0.015+0.015

All gains are positive across all 5 seeds; notably, on DINOv3-sat, Qwen3-VL-4B text adds only +0.003+0.003, confirming the gain stems from RL training. In the reward ablation (2B, ViT): Qwen3-VL-2B achieves 0.7470.747, prediction-only 0.7510.751, analysis-only 0.7580.758, and full composite 0.7680.768 (+0.016+0.016 over pred-only, +0.009+0.009 over analysis-only).

Training curves & stability (added to Appendix for TRACE-4B):

  • Stage 1 (3737 steps): Validation composite reward rises from 0.4850.485 to 0.5160.516, gated analysis score RAR_A from 0.6880.688 to 0.7510.751, and RpredR_{\text{pred}} from 0.3510.351 to 0.3600.360, with KL <0.07< 0.07 and entropy in 0.88–0.920.88\text{--}0.92 (no collapse or drift).
  • Stage 2 (3737 steps, Predictor refitted on Stage-1 texts): Composite reward plateaus at 0.517–0.5210.517\text{--}0.521 and RAR_A at 0.750–0.7590.750\text{--}0.759 (KL <0.023< 0.023), with no drop upon Predictor handoff (0.516→0.5190.516 \to 0.519). This narrow plateau (±0.002\pm 0.002) serves as our stopping criterion; final checkpoints lie within 0.0020.002 of peak validation reward in both stages.

[Q3] We added two cross-family evaluators (gpt-5.6-sol, gpt-6-astra) on the 5050 audit cases, scoring factuality, carbon relevance, and hallucination-freeness on randomized, anonymized pairs given the image, city, and month:

EvaluatorTRACE-4B PreferredTieQwen3-VL-4B Preferred
Human 1 / Human 238 / 273 / 79 / 16
gpt-5.6-sol / gpt-6-astra28 / 305 / 217 / 18
Qwen2.5-VL-72B Judge31118

The human reference uses Table 2's case-level audit scores. Strict-preference agreement is the fraction of matching directions, excluding cases with a tie from either evaluator. We use directions to avoid score-calibration differences (Human 2 scores 1.51.5 pts higher on average): Human 1 vs. 2 is 27/4127/41 (65.9%65.9\%); 72B Judge vs. human audit is 25/3625/36 (69.4%69.4\%); gpt-5.6-sol and gpt-6-astra vs. human audit are 29/4129/41 (70.7%70.7\%) and 31/4331/43 (72.1%72.1\%); and agreement between the two GPTs is 37/4337/43 (86.0%86.0\%). All five evaluators prefer TRACE-4B. On these 100100 audited texts, the 72B Judge averages 0.6860.686 (Qwen3-VL-4B) vs. 0.7540.754 (TRACE-4B), supporting the same trend as Table 2 (full tables and bootstrap CIs in Appendix H).

[Q4] Clarifications & cold-start confinement. (i) In Appendix A.1, "quantitative" was a typo for "qualitative". (ii) Teacher pseudo-analyses on the Predictor-training subset are used exclusively to initialize the cold-start Predictor (and on the Analyst-training subset only for cold-start early stopping). The Analyst is never trained on pseudo-analyses and never sees labels; subsequent Predictors are trained solely on the Analyst's outputs, and test inference uses no Teacher texts.

Negligible label values in text (excluding prompt boilerplate like "1 km" and "2022"):

Text SourcennAny NumberNumber + Emission UnitWithin ±0.5%\pm 0.5\% of Raw Ref.
Teacher pseudo-analyses19,2251.25%0.05%0.05%
TRACE-4B / Qwen3-VL-4B (test)4,94057.4% / 37.8%0.00% / 0.00%0.12% / 0.14%

Masking numbers would alter only 1.25%1.25\% of cold-start texts, and TRACE-4B falls within ±0.5%\pm 0.5\% of the raw ODIAC reference (unit match not required) no more often than Qwen3-VL-4B (0.12%0.12\% vs. 0.14%0.14\%).

Shuffled & empty controls (ViT, TRACE-2B, 5 seeds; permuting text embeddings across tiles within the same city and split before retraining the Predictor): Empty text gets 0.709±0.0120.709 \pm 0.012 (−0.058-0.058 vs. TRACE-2B's 0.768±0.0030.768 \pm 0.003) and within-city shuffled gets 0.744±0.0130.744 \pm 0.013 (−0.023-0.023 consistently across all 5 seeds), confirming reliance on tile-specific grounded content.

[Q5] Metadata & RS-VLM baselines (Appendix G.2):

Predictor InputR2R^2TRACE-2B GainTRACE-4B Gain
Image only0.716±0.0110.716 \pm 0.011+0.052+0.052+0.057+0.057
+ City/month one-hot0.752±0.0060.752 \pm 0.006+0.016+0.016+0.021+0.021
+ City/month as text0.755±0.0210.755 \pm 0.021+0.012+0.012+0.018+0.018
+ Short caption0.713±0.0120.713 \pm 0.012+0.054+0.054+0.060+0.060
+ EarthDial / GeoChat description0.709±0.0210.709 \pm 0.021 / 0.711±0.0150.711 \pm 0.015+0.058+0.058 / +0.057+0.057+0.064+0.064 / +0.062+0.062
+ TRACE-2B / TRACE-4B0.768±0.0030.768 \pm 0.003 / 0.773±0.0060.773 \pm 0.006——

TRACE consistently outperforms city/month one-hot across all seeds (+0.016+0.016 / +0.021+0.021) and improves both mean and cross-seed stability over raw city/month text (±0.021\pm 0.021 vs. ±0.003\pm 0.003/±0.006\pm 0.006). Generic captions and RS-VLMs (EarthDial, GeoChat) add no benefit over image-only. Why within-city shuffled text beats empty text and RS-VLMs: shuffling within each city retains TRACE's city-level context (Table 3), comparable to city/month one-hot (0.7440.744 vs. 0.7520.752). Conversely, although EarthDial and GeoChat receive city and month in their prompts, they rarely write them into outputs (6%6\% and 12%12\% mention the city; 4%4\% for short captions), so their descriptions express less city/month context.

[W3] Our core contribution is the problem formulation and co-evolutionary framework: (1) formulating readable spatial analysis as an interpretable latent representation for continuous regression; (2) combining downstream Predictor feedback with a gated Analysis Score (1[Rpred>0]\mathbb{1}[R_{\text{pred}} > 0]); and (3) Periodical Evolution (refitting the Predictor between stages) to prevent reward hacking.

I appreciate the authors' response and the additional experiments. The experiments address some of my previous concerns.

Although the authors conducted analyses via numerical verification and text-shuffling experiments, these do not rule out the possibility that the teacher model’s text generation was influenced by the carbon emission labels. I therefore suggest including a control setup where the teacher model operates without these labels. This would clarify to what extent the observed performance gains depend on initialization conditioned on such labels. This remains my primary concern.

While the cross-model evaluation offers some supplementary evidence, the sample size of only 50 instances is insufficient to robustly support strong claims about interpretability.

Therefore, i keep my original score.

Thank you for your thoughtful follow-up and constructive suggestions. We conducted both requested experiments, a label-free Teacher control and a 1,0001,000-sample cross-family audit, and present the results below.

1. Label-Free Teacher Control

We first clarify that providing the emission level in the Teacher prompt is an intentional design choice, and its influence is structurally confined to cold-start initialization without affecting downstream stages:

  • Design rationale: Because the Teacher (Qwen2.5-VL-72B) is an off-the-shelf VLM without task-specific fine-tuning, supplying the emission level as context helps prevent hallucinations and ensures visually grounded descriptions when initializing the cold-start Predictor (which serves as the initial reward model).
  • No downstream impact: Teacher pseudo-analyses are used exclusively to initialize this cold-start Predictor and are discarded immediately afterward. The Analyst is never trained on Teacher texts and never observes ground-truth labels, while all subsequent Predictors are trained solely on the Analyst's own outputs. Moreover, the Teacher prompt requests qualitative analysis only, and numerical audits confirm the resulting texts are virtually free of label numbers (1.25%1.25\% contain any number; 0.05%0.05\% contain a number with an emission unit).

Nevertheless, to directly address your concern and empirically verify that our gains do not depend on label-conditioned initialization, we ran the exact control you suggested. We regenerated the Teacher analyses across tiles used by the Predictor with the same Teacher model and prompt, with all emission label information removed, and retrained the cold-start Predictor under the identical protocol:

Predictor InputTest R2R^2 (mean ±\pm std, 5 seeds)
Image only0.6957±0.01350.6957 \pm 0.0135
Image + labeled Teacher text (original)0.7548±0.01650.7548 \pm 0.0165
Image + label-free Teacher text0.7549±0.00980.7549 \pm 0.0098

Removing the emission label leaves cold-start Predictor accuracy completely unchanged (0.75490.7549 vs. 0.75480.7548, with even lower cross-seed variance). This confirms both our architectural design and that label conditioning in the Teacher prompt does not drive our reported performance gains. We will incorporate this control experiment into the revised manuscript.

2. Expanded 1,0001,000-Sample Cross-Family Audit

To evaluate analysis quality at scale, we expanded our blind paired audit from 5050 to 1,0001,000 randomly sampled test tiles (200200 tiles per city across all five cities), following the exact same randomized, anonymized protocol. Two independent cross-family LLM judges (gpt-5.6-sol and gpt-6-astra) scored all 1,0001,000 pairs across factuality, carbon relevance, and hallucination-freeness:

JudgePrefers TRACE-4BTiePrefers Untuned Qwen3-VL-4B
gpt-5.6-sol55.6%55.6\%6.4%6.4\%38.0%38.0\%
gpt-6-astra53.2%53.2\%5.1%5.1\%41.7%41.7\%

Across this 20×20\times larger benchmark, both independent judges consistently prefer TRACE-4B's analyses over the untuned baseline by substantial margins (+17.6+17.6 and +11.5+11.5 percentage points, respectively).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.