acceptodds
Under review as a conference paper at ICLR 2027

Albedo Drift: What Single-Image Intrinsic Decomposition Evaluation Does Not Measure

Abstract

Albedo is illumination-invariant by definition, yet the standard evaluation of single-image intrinsic decomposition never tests invariance: WHDR on IIW scores relative reflectance within one image, so a model that bakes illumination into its albedo is never penalized. We name this failure albedo drift and introduce a protocol that measures it directly: the ratio of cross-illumination albedo variation to input variation on scenes photographed under many lighting conditions. Across three datasets and six methods we find that (i) a state-of-the-art model absorbs half of the illumination change into its albedo even within its training domain (drift ratio 0.55), and remains at 0.50 on photometrically diffuse pixels, ruling out non-Lambertian ambiguity as the cause; (ii) outside the training domain, decomposition amplifies illumination variation (drift ratio 1.5, worse than returning the input); (iii) under simulated illuminant color changes, learned models amplify chromatic variation up to ; and (iv) what determines drift is not the architecture family but the match between training and test distributions — a domain-matched diffusion model is as invariant as regression models, while a mismatched one is worse. Invariance alone is trivial (a constant albedo is perfectly invariant), so we evaluate it jointly with fidelity and observe that no existing method occupies the region that is both faithful and invariant. This corner is reachable: a 1M-parameter consistency adapter trained on held-out scenes halves drift (, sign test ) while slightly improving WHDR, generalizes across capture domains, and cannot be replicated by smoothing. Albedo drift is not a fundamental trade-off; it is an axis the field has never optimized because it has never been measured.

Primary area:
datasets and benchmarks
Keywords:
intrinsic image decomposition, albedo, illumination invariance, evaluation protocol, benchmark, image formation, consistency training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zHrQOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper propose a new metric and benchmark: albedo drift, which measures the variation of predicted albedo of intrinsic image decomposition methods under varying illuminations. The paper found modern intrinsic image decomposition is often not as robust as even classical Retinex for albedo drift. The author propose a trained U-Net base adapter refinement on albedo prediction, which is trained on multi-illumination data, to improve albedo drift. The authors found albedo drift is improved at not cost to fidelity as defined by whdr.

Critical strengths

there can be interesting points if true (although I believe the current evidence presented in the paper is too weak). For example, albedo drift is significantly worse for out of domain examples.

Critical weaknesses
  1. I have a philosophical concern regarding the paper: like the author said, there has been plenty of work which study albedo invariance within an image. The author propose another invariance of albedo which is measured across illumination. I am worried the community as a whole can keep proposing many more invariants, but nothing really specifies what correct albedo should be exactly. It is also not clear if albedo drift an extraordinary important invariance in downstream tasks. While illustrative of the albedo drift, I find the example where the author relight the albedo recovered from multiple illumination under the same new illumination to be a bit artificial, as it is not clear if this is representative for how IID will be used in downstream tasks.

  2. As a benchmark paper, the benchmark is not very exhaustive. recent methods such as "RGB↔X" or "IntrinsicAnything" are not benchmarked. Additional regression baselines such as NIID-net, or "Inverse Rendering for Complex Indoor Scenes" is also not benchmarked. I think it is difficult to draw reliable conclusions for each class of method based 1 or 2 techniques. "Findings" section of the paper. It is also not clear why Marigold-IID fails so badly on albedo drift. providing qualitative results would be really helpful.

  3. As a dataset paper, I feel the qualitative results is way too few and readers might have difficulty intuitively understanding how the albedo drift of various different methods manifest on these examples. For the only result of albedo in the main paper, figure 2, a large percentage of the visible pixels are glasses, and it is really not clear what is the correct albedo in those regions. There is also no qualitative results for albedo spectral drift at all.

  4. Albedo drift isn't really an entirely new concept. Even back in 2018, "Learning Intrinsic Image Decomposition from Watching the World" proposed to minimize albedo variations across different illuminations. The proposed albedo drift and adapter is very close to "Learning Intrinsic Image Decomposition from Watching the World".

  5. Why many fields of table 1 is left empty? e.g. drift (diffuse), drift (spectral), WHDR, relight? Completeness of the results is very important to assess the authors' conclusion of the paper

  6. minor. I do not believe eq.4 is gauge transform. please clarify.

Questions for authors

Why is albedo drift worthwhile single out as an invariance for the intrinsic image decomposition problem? How is albedo drift different from constant albedo assumption that is used by technical such as "Learning Intrinsic Image Decomposition from Watching the World" .

4 comments
AuthorsOpenReview ↗

Response to Reviewer zHrQ (1/2): the question of the review, Retinex premise, class-level finding withdrawn

Response to Reviewer zHrQ (1/2): the question of the review

Thank you for the detailed review. This comment answers your conceptual question; the next, the evidence.

Your pointWhat changed (where)
Why single out this invariance; what is "correct albedo"Below: the defining invariance, a necessary condition read with fidelity (§1, §3.2–3.4)
Relighting artificial; downstream importance unclearAgreed; kept as a proxy, limit stated (§6, Limitation 4)
Benchmark not exhaustiveRGB→X and NIID-Net added; IntrinsicAnything and Li et al. 2020 named as unevaluated (§2, Tables 1–2)
Why Marigold fails so badly; qualitative resultsWe had measured its shading; new Fig. 4 (six scenes), Fig. 5 (§4, App. C)
Fig. 2 glass pixels; no spectral qualitative resultsDiffuse-only control; probe withdrawn as a measurement (Table 1, §4.1)
Li & Snavely 2018; empty Table 1 cells; Eq. 4 "gauge"Positioning rewritten; Table 1 complete; "gauge" removed (§2, §3.2)

Why single out this invariance, and what "correct albedo" is. We do not propose a new invariant; we test the one that defines the quantity. Albedo is, by definition, the property of the surface that does not change when the light does; WHDR scores relative reflectance within one photograph under one light and cannot test it. Drift is a necessary condition, not a sufficient one: a correct albedo has zero drift (up to the scale ambiguity of §3.2 and the interreflection floor of Limitation 3), but zero drift does not make an albedo correct, which is why every result is read jointly with fidelity (WHDR, and ground-truth error where it exists) and why a constant output, perfect on drift, is rejected on fidelity. We do not claim to specify the correct albedo. We show that every learned model evaluated keeps a fifth to two fifths of the normalized change (0.193–0.441 on multilum; §3.4 treats this as an upper bound on invariance failure), the best regression model on its own training distribution, and that WHDR cannot see this. Whether that matters downstream is not established here: relighting flicker is a proxy and, as you say, an artificial one; the applications that need consistency across captures (relighting, material extraction from several photographs, editing across frames) are not tested, and Limitation 4 says no downstream task is evaluated end-to-end. The difference from Li & Snavely (2018) is one of role, not of physics: they use the same assumption as a training loss on time-lapse video; we measure how much of it current models violate after training without it, in a normalized ratio comparable across methods (its absolute scale is dataset-dependent, §3.3), read jointly with fidelity. The adapter's objective is in the same family as theirs, and §5 says so; our contribution is the measurement and its consequences, not the loss.

What the measurement adds beyond the definition. (a) Fidelity does not predict invariance: C&A v2 and Marigold-IID have WHDR 0.217 and 0.215 but drift 0.200 and 0.441; Kocsis et al. scores 0.200 at WHDR 0.297 and NIID-Net 0.193 at 0.235, both WHDR values on the 300-image subset (Finding 4, corrected). (b) The off-domain growth you singled out as interesting if true holds with the corrected numbers: C&A 0.200 → 0.595 on BigTime, Kocsis 0.200 → 0.380, RGB→X 0.264 → 0.674, Marigold 0.441 → 0.916, NIID-Net 0.193 → 0.315 (Finding 3). (c) The encoding discipline of §3.4 and its identity-in-linear control apply to any evaluation comparing a model's linear-RGB output with sRGB photographs: a no-op scores 3.86 on multilum under the submitted convention.

Two submitted claims that the correction reverses. Retinex is not "more robust than modern decomposition": under matched encoding it is less invariant than C&A (0.327 vs 0.200) and less faithful (WHDR 0.450 vs 0.217). And the class-level conclusion of the submission (Finding 4, "distribution match, not architecture, governs drift") rested on Marigold's shading value and is withdrawn; the corrected paper states its findings per model and claims only that, among the methods evaluated, no architecture family is invariant and the within-image score does not reveal which model keeps less of the light. We agree that one or two models per class cannot support class-level conclusions; the revision draws none.

What the corrected paper claims. Less than the submitted one: the median directional drift of the five learned pipelines on multilum is 0.193–0.441, the leak is structural and grows off-domain, WHDR does not predict it, and on two base models a consistency objective reduces it without WHDR degradation, while ground-truth albedo error on MIT worsens on both. Every number traces to a result file to be released with the code.

If this resolves your concerns, we would be grateful if you would reflect it in your assessment; if something remains, we would like to know what it is before the discussion closes.

AuthorsOpenReview ↗

Response to Reviewer zHrQ (2/2): Marigold failure explained, qualitative results, Table 1 filled, RGB→X and NIID-Net added

Response to Reviewer zHrQ (2/2): the evidence

Why does Marigold-IID "fail so badly"? Not as badly as we reported: the submitted number was a shading measurement. Our Marigold scripts selected the wrong output: list(pipe.target_properties).index("albedo") indexes the keys of the property dictionary and returns the shading target. Every submitted Marigold number was a shading measurement. With the albedo target and matched encoding, Marigold's median drift is 0.441 [0.370, 0.620] against 0.200 for C&A, and its full-IIW WHDR is 0.215 (C&A: 0.217). The two models therefore have nearly the same full-IIW WHDR and a two-fold difference in invariance, which is the corrected form of Finding 4. Sampling noise contributes to Marigold's drift but does not create it: five seeds on four scenes put same-image variation at 0.31 of the cross-illumination variation, and a 10-sample ensemble lowers the median from 0.441 to 0.408 (−7.5%, paired p < 1e-4). Marigold also returns near-black albedo for dark objects and, in one scene, for almost the whole scene under one light (Fig. 4); such outputs are detected by thresholding the prediction's median and counted separately (§3.2).

Qualitative results, and the glass pixels of Fig. 2. Regenerated under the matched encoding: Fig. 2 (drift structure maps on everett_kitchen2, the scene with the largest C&A drift), a new Fig. 4 (six scenes × {input, C&A, Marigold} under the two lights whose normalized inputs differ most; per-scene ratios in Appendix C) and Fig. 5 (relit crops). You are right that much of Fig. 2 is glass and specular surfaces, where the correct albedo is ambiguous; that is why the protocol has the diffuse-only control: Eq. (7) removes saturated pixels and moving highlights, and the ratio restricted to the remaining pixels is 0.206 for C&A (0.200 on all pixels), 0.423 for Marigold (0.441), 0.228 for Kocsis et al. and 0.289 for RGB→X (Table 1). Drift persists after the diffuse-only control; it is not an artifact of the non-Lambertian regions alone. The strong colour casts in the submitted panels were themselves partly an artifact of the mixed encoding. There are no qualitative spectral results in the revision either: the simulated probe is withdrawn as a measurement (§4.1), and the spectral axis is listed as unmeasured (Limitation 2).

Incomplete Table 1. Table 1 is now complete. Kocsis et al. is evaluated on multilum, MIT Intrinsics and BigTime: 0.200 [0.179, 0.210] (C&A 0.200), 0.187, and 0.380 (C&A 0.595); spectral probe 3.67 (ill-conditioned, not interpreted), WHDR 0.297 (300-image subset), diffuse-only 0.228, relighting flicker 0.20 (all with the authors' x^2.2 input linearization, general response item 4). It matches C&A's invariance at a worse WHDR, a second instance of the fidelity–invariance decoupling. The last empty cell, the WHDR of C&A 2023, is filled (0.254 on the 300-image subset).

Additional public methods. We added the RGB→X model of RGB↔X (Zeng et al., 2024), run with the authors' demo settings (prompt "Albedo (diffuse basecolor)", 50 DDIM steps) and with the pipeline's 1/2.2 output gamma inverted so that its albedo enters the metric in linear RGB like the other learned models. Results (matched encoding, medians; multilum, MIT Intrinsics and BigTime): 0.264 [0.224, 0.279] (diffuse-only 0.289), 0.277, 0.674; WHDR 0.257 on the 300-image IIW subset. It sits between Kocsis et al. (0.200) and Marigold (0.441) in drift, and it shows the same off-domain growth (0.264 → 0.674 on BigTime). We also added NIID-Net (Luo et al., 2020), the regression baseline you named, run with the authors' inference script (it consumes the encoded image, runs at a fixed 320×240 and upsamples its linear reflectance to the frame; stated in §4): 0.193 [0.181, 0.206] on multilum (diffuse-only 0.229), 0.148 on MIT Intrinsics, 0.315 on BigTime, WHDR 0.235 on the 300-image subset. It matches C&A's in-domain invariance at a worse WHDR and grows off-domain by half rather than threefold; its low internal resolution accounts for little of this (run at the frame's resolution: 0.208 on multilum, 0.324 on BigTime; Appendix A). Both models are in Tables 1–2, Fig. 1, Findings 3–4 and the appendix mapping. IntrinsicAnything and the inverse-rendering network of Li et al. (2020) remain unevaluated and are named as such in §2 and Limitation (5).

Eq. 4 and the term "gauge". You are right. The scalar ambiguity of I = R·S (and its per-pixel generalization R·f(x) · S/f(x)) is now described as a scale ambiguity; "gauge" is removed. §3.2 states what the per-image median normalization fixes and what it leaves (a global per-channel cast is additionally forgiven by the per-channel variant, reported as a control: C&A 0.205).

If this resolves your concerns, we would be grateful if you would reflect it in your assessment; if something remains, we would like to know what it is before the discussion closes.

I thank the authors for an thorough and honest revision. The authors reported 3 defects of the orignal version (linear/srgb mismatch, marigold measured shading instead of albedo, ill-conditioned simulated psectral probe). The authors therefore updated the results, and added baselines, completed table 1, added second base model all made me more confident in the authors reported numbers. My point 2, 5, 6 are resolved.

My core remaining concern is fact my philosophical concern raised in point 1. The community can keep adding more invariance dimensions, but invariance is merely a property and does not specify what is correct. The results could still be wrong even all invariants are met. The newly reported results furthers my concern. On MIT intrinsics for example, the proposed adapter reduces drift by 20% (0.197->0.157), and lowers WHDR, but the ground-truth albedo error rises by 33% (si-MSE 0.00147 -> 0.0195), and by 111%(0.0217->0.0459).

As the authors optimizing for this invariance, predicted albedo becomes much more distant from groundtruth albedo. The results in fact strongly supports that optimizing for one extra invariance could be counterproductive and new technique that evaluate albedo comprehensively is needed.

Additionally, the author fail to support why such invariance is particularly important in downstream tasks, which should be the core motivation of the paper. No discussions nor experiments or user studies are performed in this regard. The adapter proposed is also not very useful, as it optimizes an invariance that is not necessarily important in downstream use cases, while harming the error when compared to groundtruth.

Therefore, I believe the proposed metric is very limited, and do not believe the paper meets the high bar of ICLR. The paper could even misled the community toward optimizing for wrong objective if published.

Other concerns:

  1. I still do not believe the idea proposed in this paper is meaningfully different from Li & Snavely (2018), as both papers use cross illumination variance as objective for training and evaluation.

  2. I still think the qualitative results are quite limited for a computational photography paper, where the qualitative results often communications more about the properties of the systems to the reader. fig.2 is still mostly glasses. fig. 4 only has two examples. Most techniques that are presented in table one are missing. Moreover, there is no qualitative results shown for the proposed adapter technique to allow reader understand what the adapter did. There is also no MIT intrinsics qualitative results or IIW qualitative results. Finally adding various quantitative metrics (drift, WHDR, si-MSE), could help reader intuitively understand what each level of quantitative results looks like qualitatively.

  3. Given major errors was discovered in the original paper, I am not very confident about the quality of the paper can be completely addressed within one conference review cycle, which is more fast paced rather then comprehensive. There could be still undiscovered major errors in the paper, and I think the paper could benefit from going through another review cycle.

AuthorsOpenReview ↗

Response to Reviewer zHrQ (round 2)

Thank you for the second reading and for confirming that points 2, 5 and 6 are resolved. New in the PDF uploaded with this comment: a contrast-matched drift ratio (Eq. 6, requested by Reviewer uUXy), MIT ground-truth error for all operating points but one (Table 4), five qualitative figures (Figs. 5–9) and the Li & Snavely distinction (§2).

1. "Invariance is a property, not correctness; optimizing it can be counterproductive." We agree, and the new measurements sharpen your point. Under the contrast-matched form (each frame standardized to zero mean and unit spatial std; identity = 1), the signed change of medians for the C&A adapter is −28.5% under Eq. 5 but −6.9% [−15.6%, +0.4%] under Eq. 6; for the NIID-Net adapter −38.5% vs. −5.9%; for the λ=3 fine-tune −39.9% vs. +11.7% (worse than the base) at 0.54 of the base's spatial contrast. On MIT every one of the 12 refinements of C&A has a higher mean ground-truth error (adapter +32.8%, λ=3 fine-tune +80.0%, NIID-Net adapter +111.3%; your 0.00147 should read 0.0147), and the change in drift ratio and the change in error are rank-correlated at ρ = −0.79 (p = 0.002). At their headline operating points the consistency objectives lower the drift ratio without worsening full-IIW WHDR mainly by flattening the albedo; after contrast matching the adapters show no resolved improvement, the fine-tune is worse, and the albedo is further from the truth.

We have therefore withdrawn "the axis is improvable at no fidelity cost" (Note on this revision, §1, §2, §5, Limitation 4, conclusion, Fig. 1 caption). The adapter and the fine-tune remain as probes showing that Eq. 5 can be moved while WHDR stays put, and how — evidence that WHDR alone is not a sufficient fidelity axis either. Abstract and conclusion now state your recommendation: report Eq. 5 with its contrast-matched form and ground-truth error where available; do not optimize invariance alone.

Where we still differ: that a necessary condition can be gamed when optimized in isolation is not an argument against measuring it. WHDR has the same property — it is blind to a flattening that nearly halves the albedo's contrast (Table 4, Fig. 9) — and our ground-truth check covers 20 lab objects. Two-light photographs give a controlled test of a necessary property under the stated image-formation assumptions (measured drift is an upper bound, §3.4), which the learned models fail at 0.193–0.441 before any optimization (0.391–0.711 contrast-matched).

2. Downstream importance. Not established: no downstream task end-to-end, no user study (Limitation 4). The relighting flicker of §6 is a proxy that inherits the flattening. Applications that reuse albedo across captures (relighting, material extraction from multi-view or time-lapse captures, video editing) are exposed to the change we measure — motivation, not a demonstrated outcome.

3. Li & Snavely (2018). We re-read the paper. Their cross-frame reflectance consistency is a training loss (their §5.2); their evaluation is WHDR on IIW, AP on SAW and MSE/LMSE/DSSIM on MIT (their §6); no cross-illumination consistency of any output is reported. They train one model with the signal; we measure it, under a stated encoding, normalization, identity control and contrast-matched form, on current models none of which was trained with it, and show that within-image scores do not predict it (§2). Our adapter's objective is of their family (§2, §5); their evaluation did not measure cross-illumination instability or the contrast-matched diagnostic.

4. Qualitative results. Appendix D now has Figs. 5–6 (all Table 1 methods and the three refined models, six scenes, two lights, light-b panels annotated with both drift ratios), Fig. 7 (base vs. refined albedo, |A_a − A_b| maps, signed residuals, metrics per row), Fig. 8 (five MIT objects by a fixed rule, ground truth and every method with a MIT value, drift / contrast-matched drift / si-MSE) and Fig. 9 (four IIW images by a fixed rule, every method, per-image WHDR). The flattening is visible, and so is its invisibility to WHDR. Fig. 2 keeps everett_kitchen2, the scene with the largest C&A drift (0.50); its glassware is why the diffuse-only value (C&A 0.206) is reported beside it; the new figures cover five other scenes. The previous Fig. 4 had six scenes but only two models and moved to the appendix.

5. Another review cycle. You and Reviewer uUXy identified the spectral-probe problem; checking it, we found the encoding and target-selection defects, and an interface audit found the Kocsis input transform. Every number comes from one macro pipeline over the result files, with identity controls and recomputation against the archived values (Appendix B), to be released with the code. The contrast-matched analysis found no new pipeline defect; it found a limit of the claim, which the paper now states. These checks do not exclude undiscovered errors; the revision documents the corrections, the checks and the remaining limitations.

Reviewer_BVbSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper examines whether single-image intrinsic decomposition produces consistent albedo estimates when illumination changes. It defines a drift ratio based on normalized cross-illumination variation and evaluates it alongside fidelity measures, since invariance alone admits constant predictions. Experiments cover three datasets and six methods or baselines, including controls for sampling variability and normalization, as well as simulated illuminant-colour changes. A small consistency adapter reduces directional drift from 0.547 to 0.277 while improving full-IIW WHDR from 0.217 to 0.210. Additional experiments examine transfer to other datasets, smoothing baselines, and relighting stability.

Critical strengths
  1. The paper identifies an important gap between benchmark performance and the physical meaning of albedo. A model can perform well on within-image reflectance judgments while its predicted albedo still changes substantially with illumination. Demonstrating this in existing methods is a useful insight: it shows why a good WHDR score alone is insufficient to establish that a model has recovered an intrinsic scene property.

  2. The adapter results show that drift can be reduced without an observed loss of prediction quality. All 30 test scenes improve, and full-IIW WHDR decreases slightly. On MIT Intrinsics, both drift and ground-truth albedo error decrease. This is useful evidence that the proposed evaluation identifies a problem that can be addressed through training, with benefits beyond the original capture setup.

Critical weaknesses

The adapter ablations mix evaluation sets and training budgets. Figure 3 reports full-IIW WHDR for the base model and main adapter, but uses a 150-image subset for the alternative anchor weights and albedo-only variant. The main adapter is also trained for 20k steps, compared with 10k for the other variants. These differences prevent attributing the results cleanly to the anchor weight or image conditioning. The main adapter’s improvement remains supported by the shared full-IIW evaluation, but the claimed preferred operating point and fidelity benefit of image conditioning require matched comparisons.

Questions for authors
  1. Given the per-channel normalization used in the spectral experiment, could you specify the RGB space in which the drift is computed and report summary statistics of the per-scene numerators and denominators? Do any scenes have very small denominators, and, if so, how are they handled? This would help interpret the reported 7–10× amplification.

  2. Could you report the base model and main λ=1 adapter results on the same 150-image IIW subset used for the ablations? If a 10k-step checkpoint of the main adapter is available, could you also check whether the claimed knee in the trade-off curve and the fidelity benefit of image conditioning hold when the training budget is matched?

2 comments
AuthorsOpenReview ↗

Response to Reviewer BVbS: RGB space and per-scene denominators, matched-subset and matched-budget ablations, MIT error reversed

Response to Reviewer BVbS

Thank you. Your observation about the normalization was correct (the general response describes what it led us to); your ablation requests have been implemented.

Your pointWhat changed (where)
RGB space; per-scene numerators and denominators; small denominatorsStated; all 30 scenes tabulated; nothing thresholded (§3.4, §4.1, Table 5)
Ablations mix evaluation sets and budgetsAll rows on one 300-image subset, adapter ablations also on the 150-image subset; retrained λ=1 at 10k and 20k (§5, Table 4)
150-image values; knee and image conditioning at matched budgetBoth reported and hold (Table 4)
Strength noted: MIT ground-truth error improvesReversed by the correction, +32.8% worse (§5, Limitation 4)

RGB space, numerators and denominators. Drift is computed in the encoded (sRGB) domain of the photographs: the model's linear albedo is re-encoded with the inverse of its own linearization (exponent 1/2.2) and compared with the photograph as stored, after per-image median normalization of luminance or of each channel (Eqs. 5–6, §3.4). For the simulated probe the per-scene denominator has median 0.0094; for real two-illuminant captures with the same normalization it is 0.1546 (16.5× larger). Every scene has a small denominator: the 30 values lie between 0.0039 and 0.0151, all below the real-pair median (Table 5). None is thresholded or excluded; no probe-denominator threshold is applied, the ratio is reported and declared ill-conditioned (the near-black detector of §3.2 acts on the prediction's median, not on the denominator). Under the matched encoding C&A's numerator has median 0.0219 and its ratio is 2.70 [2.10, 3.13]; the submitted 6.77 used the mixed encoding over the same near-zero denominator, and Marigold's submitted probe value measured shading (general response, items 1–2). The probe is retained as a documented negative result; the spectral claims are withdrawn.

Matched comparison of base and adapter. The submitted ablations mixed evaluation sets (full IIW for the base and λ=1, a 150-image subset for the other variants, as its Fig. 3 caption stated). Every ablation row in the revision (Table 4) is on one deterministic 300-image subset of IIW, base included: base 0.2085, adapter λ=1 0.2087 (+0.1%), λ=3 −1.5%, λ=0.3 +20.0%, albedo-only input +2.3%. As you asked, the same rows are given on the submitted ablations' 150-image subset: base 0.2100, λ=1 (20k) 0.2100 (0.0%), λ=3 0.2044 (−2.7%), λ=0.3 0.2487 (+18.4%), albedo-only 0.2168 (+3.2%). The base and the λ=1 adapter also have full-set values (0.2174 → 0.2100, −3.4%), as do the fine-tuned networks; the table shows all three columns.

Training budget (10k vs. 20k steps). We retrained the λ=1 adapter and evaluated it at 10k and 20k steps: median drift 0.141 and 0.145 (original 20k checkpoint: 0.143), WHDR on the 300-image subset 0.2058 and 0.2007 (−1.3% and −3.7%), on the 150-image subset 0.2084 and 0.2029 (−0.8% and −3.4%). At the matched 10k budget on the same 300-image subset the image-conditioned adapter is at 0.141 drift and −1.3% WHDR, the albedo-only variant at 0.149 and +2.3%: the fidelity benefit of image conditioning holds at matched budget, and the drift reduction does not depend on it. The budget does not separate the rows; the retrained run gives similar results to the original checkpoint (one retraining, two checkpoints).

Corrected trade-off curve. Fig. 3 is redrawn on one IIW subset; the knee is unchanged: the adapter's anchor weight traces one curve (λ=0.3: drift 0.024 at +20.0% WHDR; λ=1: 0.143 at +0.1%; λ=3: 0.188 at −1.5%); consistency fine-tuning of the base network traces another (λ=1: 0.047 at +81.1% WHDR on the same 300-image subset, +66.2% on the full set; λ=3: 0.120 at +2.4%, full set −0.5%; λ=10: 0.173 at +2.8%, full set +1.3%); smoothing moves along neither (bilateral: drift unchanged; Gaussian blur: −6.9% drift for +22.8% WHDR); neither learned curve dominates the other.

The MIT result you listed as a strength has reversed. The submitted statement that the adapter lowers ground-truth albedo error on MIT Intrinsics rested on linear files fed to an sRGB-expecting model; with the files encoded before inference the adapter reduces drift there (−20.2%) but worsens si-MSE from 0.0147 to 0.0195 (+32.8%, p < 1e-4). Limitation (4): the axis is improvable, the refined albedo is not shown to be more accurate.

Pre-registered criterion. The adapter reduces drift by 30.5% of means and 28.5% of medians (97% of scenes improve), at no full-IIW WHDR cost. The criterion (≥30% reduction, WHDR within +10%) is met on means and missed on medians; the abstract gives both reductions and §5 states which part is met. We did not adjust the threshold.

If this resolves your concerns, we would be grateful if you would reflect it in your assessment; if something remains, we would like to know what it is before the discussion closes.

Thank you for your response. I have no further major concerns and stand by my recommendation. And I have updated the AI-use documentation field.

Reviewer_uUXyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Albedo is illumination-invariant by definition, yet standard evaluation of single-image intrinsic decomposition (WHDR on IIW) scores only relative reflectance within a single image and never tests this invariance. The paper calls this failure albedo drift and measures it as the ratio of cross-illumination albedo variation to input variation across scenes photographed under many lighting conditions. Across three datasets and six methods, a state-of-the-art model absorbs a large share of illumination change into its albedo even in-domain, amplifies illumination outside its training domain, and amplifies illuminant color changes severalfold. Drift is driven by training–test distribution match rather than by architecture family. A small consistency adapter roughly halves drift while slightly improving WHDR, showing that drift is not a fundamental trade-off but an axis never optimized because it was never measured.

Critical strengths
  • The invariance that defines albedo was not tested by the existing evaluation, and this paper tests it. The proposed drift ratio is simple and easy to interpret (returning the input unchanged scores 1) and is reported jointly with fidelity so that it cannot be gamed by a constant output.

  • The controls are careful. Changing the random seed, ensembling, the normalization method, the input color space (linear RGB), or restricting to diffuse pixels does not eliminate the drift, so none of these factors explains it. Simple smoothing (blur, bilateral filter) also fails to match the adapter’s gain.

  • The findings are informative. Drift is larger outside the training domain, and invariance to lighting direction and invariance to illuminant color are distinct axes.

Critical weaknesses
  • The large spectral drift (7–10×) may be partly inflated by normalization. Per-channel normalization is applied, and by the definition of Eq. (6), the input in the denominator appears to be normalized the same way. Because an illuminant color change is nearly a per-channel scaling, the denominator may become very small, inflating the ratio. The paper does not report the denominator’s value or results without this normalization.

  • The adapter is compared only with smoothing filters, not with existing consistency-based training, and is tested on a single base model. The main result is derived from the same dataset used for training (different scenes), and on BigTime the median drift remains above that of returning the input, which weakens the claim that the adapter generalizes across capture domains.

Questions for authors
  • How does the adapter compare with existing consistency-based training under the same conditions, and does it work with other base models?

  • Since drift is only weakly correlated with ground-truth albedo error (Spearman 0.18), can the authors demonstrate that it matters beyond the relighting flicker metric, e.g., in a user study or a downstream editing task?

3 comments
AuthorsOpenReview ↗

Response to Reviewer uUXy: probe denominator confirmed, fine-tuning and second base model added, in-distribution scope stated

Response to Reviewer uUXy

Thank you for a careful review. Your point about per-channel normalization was correct and started the audit in the general response.

Your pointWhat changed (where)
Spectral drift inflated by normalization; denominator and unnormalized results missingConfirmed; denominators, numerators, luminance-normalized values per scene; 7–10× withdrawn (§4.1, Table 5)
Adapter vs consistency-based training; single base modelConsistency fine-tuning of the base network at λ ∈ {1, 3, 10}; adapter retrained on a second base model, NIID-Net (§5, Appendix B, Limitation 4)
Main result in-distribution; BigTime drift above 1Stated; the >1 value was the encoding artifact; corrected 0.532 vs base 0.595 (§4, §5, Limitation 7)
Weak correlation with ground-truth error; evidence beyond flickerCorrelation corrected (0.32); no user study or editing task, stated as not established (§2, §6)

Per-channel normalization and the spectral probe. The simulated sweep's denominator has median 0.0094 across the 30 scenes, against 0.1546 for real two-illuminant pairs (LSMI, same normalization), 16.5× smaller: per-channel median normalization removes a global gain almost exactly. The 7–10× figures are a small numerator over a near-zero denominator, computed under the mixed encoding (Marigold: on the shading target), and are withdrawn. §4.1 reports numerator and denominator (C&A 2.70 [2.10, 3.13], median numerator 0.0219), Table 5 per scene. Without it (luminance normalization, which does not forgive the per-channel gain) the denominator is 0.0785 (8.4× larger) and the median ratios are C&A 0.28, Kocsis 0.37, RGB→X 0.47, Marigold 0.83, Retinex 0.07, adapter 0.19 (identity 1.00 under both). We conclude nothing from a simulated probe. On the 30 real LSMI pairs C&A's ratio is 0.63 [0.53, 0.77]; the spectral axis remains unmeasured (Limitation 2).

Fine-tuning the base model versus the adapter. New in §5 and Fig. 3: C&A's albedo network (105M-parameter final stage; earlier stages frozen) was fine-tuned with the adapter's objective on the same scenes at λ ∈ {1, 3, 10}: median drift 0.200 → 0.047 / 0.120 / 0.173 at full-IIW WHDR 0.3612 / 0.2162 / 0.2201 (+66.2% / −0.5% / +1.3% vs base 0.2174; adapter 0.2100, −3.4%). λ=3 is more invariant at unchanged WHDR, the adapter more faithful; λ=1 collapses fidelity. Second base model. The adapter was retrained with the same recipe (λ=1, 20k steps, seed 0, same scenes) on frozen NIID-Net (Appendix B). Drift falls 0.193 → 0.119 [0.108, 0.128] on multilum (−38.5% of medians, every scene), 0.315 → 0.199 on BigTime, 0.148 → 0.051 on MIT; WHDR (300-image subset) improves 0.2351 → 0.2115 (−10.0%), but MIT ground-truth error worsens (si-MSE 0.0217 → 0.0459, +111.3%, 19 of 20 objects), as for C&A (+32.8%). The MIT reduction exceeded our pre-registered range, so we checked how it arises: the refined albedo has less spatial contrast than its base (median std ratio 0.75 multilum, 0.51 BigTime, 0.35 MIT; C&A adapter 0.95 and 0.74 on the first two): part of the gain is contrast loss, which WHDR does not penalise and ground-truth error does. A zero-initialised adapter reproduces NIID-Net's drift and WHDR exactly. Fine-tuning remains on C&A v2 only (Limitation 4).

In-distribution result and BigTime. You are right: C&A v2's supervision derives from the multilum train split; §4 now calls 0.200 a best-case, in-distribution number (Limitation 7). On BigTime the submitted medians above 1 were the encoding artifact (a no-op scores 2.86 there under the submitted convention); corrected, the adapter's BigTime drift is 0.532 vs the base's 0.595 (−10.5%, p < 1e-4; −26.0% on HDR renders, −20.2% on MIT): transfer of a consistency objective across capture setups, not generalized invariance.

Downstream evidence. The correlation you quote is corrected too: with matched encoding, drift correlates with ground-truth albedo error at Spearman 0.32 over the 20 MIT objects (−0.20 under the submitted input protocol); still weak, no conclusion drawn. Agreed: relighting flicker is a proxy, not an application study. Of the controls you list, the linear-RGB-input control was not recomputed and is omitted, as are the submission's test-time-augmentation and reference-frame controls (Table 6 caption); the others were recomputed and do not remove the drift. (i) The metric is reported as defined, with a diagnosis of why Marigold's value (7.36) is an order of magnitude above its drift ratio: the reference shading I/A diverges where the predicted albedo approaches zero, and 81% of Marigold's summed variation comes from 1% of its pixels (§6); metric and threshold unchanged. (ii) We ran no user study or editing task; Limitation 4 says no downstream task is evaluated end-to-end.

If this resolves your concerns, we would be grateful if you would reflect it in your assessment; if something remains, we would like to know what it is before the discussion closes.

Thank you for the thorough revision. The spectral-probe concern, the missing comparison with consistency-based training, and the single-base-model concern have been addressed. In light of the revision, I have updated my strengths below.

[Strengths (revised; strikethrough = removed, bold = changed)]

  • The invariance that defines albedo was not tested in the existing evaluation, and this paper tests it. The proposed drift ratio is simple and easy to interpret (returning the input unchanged scores 1 under matched encoding) and is reported alongside fidelity so that it cannot be gamed by a constant output.

  • The controls are careful. Changing the random seed, ensembling, the normalization method, the input color space (linear RGB), or restricting to diffuse pixels does not eliminate the drift, so none of these factors explains it. Simple smoothing (blur, bilateral filter) also fails to match the adapter's gain. [Removed the linear-RGB control after the author response: the authors did not recompute it and omitted it from the revision.]

  • The findings are informative. Drift is larger outside the training domain (C&A: 0.200 → 0.595), and invariance to lighting direction and invariance to illuminant color are distinct axes. [Removed the second claim: the spectral axis is not measured in the revision (Limitation 2).]

[Remaining concerns.]

  • The adapter does not yield more accurate albedo. It increases MIT ground-truth error (+32.8% on C&A, +111.3% on NIID-Net). The authors report that part of the gain is a loss of albedo contrast, which WHDR does not penalize. Limitation 4 states that the refined albedo is “not shown to be more accurate.” The abstract says “no WHDR cost” and that the axis is improvable but omits this, so it should be added.

  • The abstract overstates fine-tuning. “Reduces drift further at an equally small WHDR change” holds only for λ=3. At λ=1, WHDR degrades by +66.2% (Table 4).

  • Evidence that drift matters is still missing. Drift correlates only weakly with ground-truth error (Spearman 0.32, 20 MIT objects). Relighting flicker is acknowledged as a proxy, and no downstream task is evaluated (Limitation 4).

[Questions.]

  • Can drift be reported alongside a contrast-controlled variant or a ground-truth error, so that drift reduction can be distinguished from contrast loss?

My concerns are partially resolved, and the score reflects the submitted revision. To address my main soundness concern, please report drift across all 30 scenes after normalizing each prediction by its own spatial standard deviation (contrast-matched), along with the MIT ground-truth error.

AuthorsOpenReview ↗

Response to Reviewer uUXy (round 2): contrast-matched drift on all 30 scenes and MIT ground-truth error

Thank you. The control you asked for changes what the paper can claim. Everything below is in the PDF uploaded with this comment (§3.3 (iv), Eq. 6, Appendix B/Table 4, abstract, §1, §5, Limitation 4, conclusion). The points you list as addressed stay as they were: the spectral probe remains withdrawn and the spectral axis unmeasured (Limitation 2); the seed, ensemble, normalization and diffuse-only controls are unchanged; the submitted linear-RGB-input control remains omitted (Table 7).

1. Contrast-matched drift on all 30 scenes (Table 4). Definition: your normalization with the mean also removed — each frame, prediction and photograph alike, is standardized over the mask (zero mean, unit spatial std) before the temporal std is taken; identity scores exactly 1.000. Dividing by the spatial std alone lets a frame's level float with its contrast, and that induced level change dominates the numerator (C&A 0.62 against identity 1; the λ=0.3 adapter 1.99), so we report the standardized form and keep the std-only values in the result files. Medians over 30 scenes, luminance (per-channel values in Table 4):

  • Base models (Eq. 5 → contrast-matched): C&A v2 0.200 → 0.407 [0.375, 0.421]; Marigold 0.441 → 0.391; RGB→X 0.264 → 0.478; Kocsis 0.200 → 0.711; NIID-Net 0.193 → 0.640; Retinex 0.327 → 0.708. Spatial contrast relative to the photograph: Marigold 0.99, C&A 0.50, RGB→X 0.51, NIID-Net 0.32, Kocsis 0.28. The ranking changes: Kocsis (equal to C&A under Eq. 5) and NIID-Net (lowest under Eq. 5) are the least stable per unit of contrast, and C&A and Marigold become similar, with overlapping intervals (Finding 4 now says so).
  • Adapter on C&A: 0.143 → 0.379, a change of −6.9% of medians [−15.6%, +0.4%], p = 0.230 (Eq. 5: −28.5%); its spatial contrast is 0.85 of the base's. Retrained adapters −6.4% / −5.8%; albedo-only −5.8%; λ=0.3 adapter (Eq. 5: 0.024) −3.4% at 0.13 of the base's contrast.
  • Adapter on NIID-Net: −5.9% [−14.2%, +5.8%], p = 0.344 (Eq. 5: −38.5%); contrast 0.70.
  • Fine-tunes of C&A: λ=3 +11.7% [+6.3%, +17.0%], p < 1e-4, at 0.54 of the base's contrast (Eq. 5: −39.9%); λ=1 +26.3% at 0.20; λ=10 +0.2% at 0.88. Gaussian blur −3.7%, bilateral −0.7%: smoothing fails to match the adapter's Eq. 5 reduction, but after contrast matching the difference between them largely disappears.

Reading: the reductions are mostly shrinkage toward the mean; the two headline adapters show no statistically resolved improvement after contrast matching, and the λ=3 fine-tune is worse. The residual maps of Fig. 7 show it (dark regions brightened, bright regions darkened).

2. MIT ground-truth error (Table 4; encoded-input protocol, si-MSE of linear albedo vs. linear reflectance, 20 objects; every operating point except the 10k NIID-Net checkpoint). Base C&A 0.0147: adapter 0.0195 (+32.8%, 20/20 objects worse); λ=3 adapter 0.0150 (+1.9%); λ=0.3 0.0454 (+209.2%); real-pair adapter 0.0349; fine-tune λ=1/3/10 0.0512 / 0.0264 / 0.0160 (+248.8% / +80.0% / +8.8%). NIID-Net 0.0217 → adapter 0.0459 (+111.3%). All 12 refinements of C&A have a higher mean error than the base, and the change in drift ratio and the change in error are rank-correlated at ρ = −0.79 (p = 0.002): the larger the fall in the ratio, the larger the rise in error. On MIT the contrast-matched drift is higher than the base's in its point estimate for every refinement (adapter +14.2%, λ=3 fine-tune +21.2%; several intervals include zero). Over the 20 objects, Spearman between drift and si-MSE is 0.32 for Eq. 5 (your value) and 0.57 for the contrast-matched form — a moderate association on a small sample. Marigold has the lowest MIT error (0.0093) and the lowest contrast-matched drift there (0.365).

3. Abstract and claims. The abstract now states the ground-truth-error increase (+32.8% adapter, +80.0% λ=3 fine-tune), restricts the fine-tuning sentence to λ=3 (λ=1: WHDR +66.2%, Table 5) and states: "the reductions are mostly a flattening of the albedo, not a more stable one. Drift is therefore a necessary condition to report beside fidelity and ground-truth error, not an objective to optimize alone." The claim that the axis is improvable at no fidelity cost is withdrawn (Note on this revision, §1, §2, §5, Limitation 4, conclusion, Fig. 1 caption).

4. Evidence that drift matters. None was added: no downstream task is evaluated end-to-end and no user study was run (Limitation 4). Relighting flicker (§6) is a proxy that inherits the flattening; it does not establish downstream utility. Applications that reuse albedo across captures (relighting, material extraction from multi-view or time-lapse captures, editing in video) are exposed to the change we measure; this is motivation, not a demonstrated outcome. What the new material does show is that WHDR registers neither the drift nor the flattening.

A control that could have falsified a claim did so; the paper now says so.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Author response (1/2): corrections prompted by the reviews, recomputed results, and what the corre

Author response (1/2): corrections prompted by the reviews, recomputed results, and what the corrected paper shows (revised PDF uploaded)

Summary for the Area Chair and reviewers. Reviewers uUXy and BVbS observed that our simulated spectral probe has a near-vanishing denominator. Checking it we found two further defects of ours (linear-RGB albedo compared with gamma-encoded photographs; Marigold-IID's shading output measured in place of its albedo) and, in a later interface audit, one input-linearization discrepancy for Kocsis et al. We recomputed every retained experiment. The research question, the drift-ratio definition and the submission's datasets and metrics are retained (LSMI's real two-illuminant pairs and §3.4 are added); two evaluation-code corrections propagate through all numbers, models and controls were added at the reviewers' request, and the claims are narrower, with withdrawn claims named in the text. Each point in the reviews is answered in our replies, with new results where asked and a plain statement where we could not provide them. Appendix A maps submitted values to corrected ones.

1. Encoding mismatch (ours). The three learned models of the submission return albedo in linear RGB (C&A's pipeline linearizes its input with exponent 2.2; Marigold-IID's prediction_space is "linear"; Kocsis et al.'s loader linearizes all targets); the submitted protocol compared it with the gamma-encoded photograph. A "model" that merely returns the linearized photograph scores 3.86 on multilum, 10.13 on the HDR renders, 5.86 on MIT Intrinsics as stored (2.46 with inputs encoded before inference) and 2.86 on BigTime under that convention: part of every submitted learned-model ratio was an artifact of mixed encodings. Corrected protocol: the albedo is re-encoded with the inverse of the model's own linearization (exponent 1/2.2) and compared with the encoded photograph (new §3.4; linear-domain convention in the appendix).

2. Marigold-IID target selection (ours). Our Marigold scripts indexed the keys of the pipeline's property dictionary and so read the shading target instead of albedo. Every submitted Marigold number (drift 3.185, ensemble 3.153, BigTime 1.748, MIT 1.902, spectral 9.83, WHDR 0.609, diffuse-only, relighting, seed control) measured shading; all are withdrawn.

3. Simulated spectral probe (uUXy, BVbS). The reviewers are right. Per-channel normalization removes a global gain almost exactly, so the probe's denominator is a residue of sRGB nonlinearity and clipping: 0.0094 (median), against 0.1546 for real two-illuminant photographs of the same kind of scenes (LSMI, 30 scenes with the largest chromaticity change), 16.5× more. The 7–10× "spectral amplification" is ill-conditioned and withdrawn as a finding; the probe is kept as a documented negative result (per-scene numerators and denominators in Table 5). On those 30 real pairs C&A's per-channel ratio is 0.63 [0.53, 0.77]; the spectral axis itself remains unmeasured in this paper (Limitation 2).

4. Kocsis et al. input linearization (ours, found while auditing). After fixing 1–3 we audited every model interface against its source code (interface cards, to be released with the code). Intrinsic Image Diffusion linearizes its input with x^2.2; our Kocsis runs had used the exact sRGB curve, which differs below an encoded value of 0.3. We re-ran every Kocsis experiment with the authors' transform; the sRGB-input run is a sensitivity check (multilum 0.195 → 0.200, WHDR-300 0.293 → 0.297; all datasets in Appendix A). C&A, Marigold-IID, RGB→X and NIID-Net match their authors' conventions.

What we did. Every retained experiment was recomputed with matched encoding and the correct target on the same scenes, frames, resolution, seeds and masks (the Marigold seed control adds one scene); 30-scene bootstrap 95% CIs (10,000 resamples) accompany the headline medians; §5 reports mean and median reductions; Appendix A separates submitted means from corrected medians. New since submission, in response to the reviews: Kocsis et al. recomputed on multilum with its empty Table-1 cells filled and newly evaluated on MIT Intrinsics and BigTime; C&A 2023's WHDR (Table 1's last empty cell); two more learned models on multilum, MIT Intrinsics and BigTime: the RGB→X model of RGB↔X (Zeng et al., 2024; 0.264 [0.224, 0.279], 0.277, 0.674; WHDR 0.257 on the 300-image subset) and the regression network NIID-Net (Luo et al., 2020; 0.193 [0.181, 0.206], 0.148, 0.315; WHDR 0.235); consistency fine-tuning of the base network at λ ∈ {1, 3, 10}; a retrained λ=1 adapter at 10k and 20k steps; the adapter on a second base model, NIID-Net (Appendix B); every ablation on one 300-image subset and the submitted 150-image subset; a Marigold seed control (4 scenes × 5 seeds) and 10-sample ensemble on the albedo target; MIT Intrinsics with files encoded to sRGB before inference. Nothing was dropped for moving.

(continued in the next comment)

AuthorsOpenReview ↗

Author response (2/2): old → new, withdrawn and kept claims, what the corrected paper contributes

Author response (2/2): old → new, withdrawn and kept claims, what the paper contributes

Old → new (median drift, matched encoding; submitted multilum values were means under the mixed convention, BigTime medians):

  • C&A v2: 0.547 → 0.200 [0.192, 0.228] (mean 0.224; per-scene range 0.08–0.50).
  • Marigold-IID: 3.185 (shading) → 0.441 [0.370, 0.620]; WHDR 0.609 (shading) → 0.215.
  • Kocsis et al.: 0.442 → 0.200 [0.179, 0.210]; WHDR 0.297 (300 images; authors' x^2.2 input, item 4 above).
  • Retinex (SSR): 0.344 → 0.327 (encoded domain, unaffected; 0.344 mean, 0.327 median).
  • C&A + adapter: 0.277 → 0.143 [0.137, 0.160]; reduction −49% → −30.5% (means) / −28.5% (medians), 97% of scenes improved; full-IIW WHDR 0.217 → 0.210.
  • RGB→X (new, Reviewer zHrQ's request): 0.264 [0.224, 0.279]; WHDR 0.257 (300 images). NIID-Net (new, regression): 0.193 [0.181, 0.206]; WHDR 0.235 (same subset).
  • Off-domain (BigTime): C&A 1.514 → 0.595 [0.499, 0.719]; Marigold 1.748 (shading) → 0.916; Kocsis → 0.380; RGB→X → 0.674; NIID-Net → 0.315.
  • Spectral probe (pc): C&A 6.77 → 2.70 [2.10, 3.13]; denominator 0.0094, ill-conditioned, withdrawn.

Withdrawn: "SOTA albedo absorbs half of the illumination change" (a fifth); off-domain "amplification" (>1); "7–10× spectral amplification" and "universal spectral fragility"; Finding 4 "training-distribution match governs drift" (the two diffusion models are no longer 6× apart); "Retinex is more invariant than the learned model" (less invariant and less faithful); "sampling noise explains 4% of Marigold's drift" (same-image sampling variation is 0.31 of the cross-illumination variation); "the adapter improves MIT ground-truth error" (it worsens it, +32.8%).

Kept, with corrected magnitudes: drift is real (median multilum drift of the five learned pipelines, C&A's two weight versions counted once: 0.193–0.441), structural (it persists on diffuse-only pixels and HDR-linear renders: C&A 0.200 → 0.206 and 0.217), larger off-domain (3× for C&A), not predicted by fidelity (C&A and Marigold: WHDR 0.217 vs 0.215, drift 0.200 vs 0.441; Kocsis and NIID-Net match C&A's drift at a worse 300-subset WHDR), and reducible on both base models tested: on C&A v2 the adapter meets the pre-registered criterion on means (≥30% reduction, WHDR within +10%) and misses it on medians, fine-tuning at λ=3 reaches 0.120 at unchanged full-IIW WHDR (−0.5%) vs the adapter's 0.143 at −3.4% (neither dominates); on NIID-Net the same recipe gives 0.193 → 0.119 (−38.5%) at WHDR −10.0%; MIT ground-truth error worsens on both (+32.8%, +111.3%).

Revised PDF. Main text ends on page 10; the OpenReview abstract and TL;DR match it. §1–2 rewritten; new §3.4 (encoding discipline, identity-in-linear control); Table 1 adds the mixed-convention column and the RGB→X and NIID-Net rows and fills its last empty cell (C&A 2023's WHDR, 0.254); Table 2 adds the identity-in-linear row; Findings 1–4 restated; §4.1 reports the probe's numerators and denominators (per scene in Table 5); §5 adds fine-tuning, retrained adapters, the second base model and the matched-subset ablation table (Table 4; 150-image column); §6 adds the relighting-metric diagnostic; Limitations cover the scope of the reduction (adapter on two base models, fine-tuning on one; refined albedo not shown more accurate) and the unmeasured spectral axis. The AI-use statement records that the defective code was AI-assisted and author-reviewed; the authors take responsibility. Reviewers and the AC may disregard revisions that differ significantly from the original; for that judgement: the research question, drift-ratio definition, datasets and metrics are the submission's (LSMI's real pairs and §3.4 added); the revision corrects what the evaluation code computes, lets the numbers follow, adds the models and controls the reviews asked for, and narrows the claims.

What the corrected paper contributes. (i) A measured, reproducible answer to a question WHDR cannot ask, how much of an illumination change survives in a model's albedo: a fifth to two fifths of the normalized change on multilum for the five learned pipelines (0.193–0.441; an upper bound on their failure, §3.4), three fifths off-domain for C&A v2 (0.595). (ii) The controls needed to measure it: matched encoding with the identity-in-linear control, diffuse-only masking, HDR-linear re-rendering, seed and ensemble controls, and the documented failure of the simulated probe. (iii) Evidence that fidelity does not predict it: C&A v2 and Marigold-IID: WHDR 0.217 and 0.215, drift 0.200 and 0.441; Kocsis et al. and NIID-Net match C&A's drift at WHDR 0.297 and 0.235 (300 images). (iv) On two base models a consistency objective reduces it without WHDR cost (C&A v2: adapter 0.200 → 0.143, fine-tuning λ=3 to 0.120; NIID-Net: 0.193 → 0.119), but the adapters worsen MIT ground-truth error on both bases.

We reply to each reviewer below and will answer further questions.

AuthorsOpenReview ↗

Second revision (discussion period): contrast-matched drift, ground-truth error for every operating point, narrowed claims

A new PDF is uploaded with this comment. It responds to the follow-up comments of Reviewers uUXy and zHrQ and narrows the paper's claims. Changes, with locations:

  1. Contrast-matched drift (new, requested by uUXy). §3.3 (iv) and Eq. 6 define a form of the drift ratio in which every frame is standardized to zero mean and unit spatial contrast before the cross-illumination variation is taken (identity = 1; a reduction obtained by flattening the albedo cannot appear in it). Table 1 gains two columns (Drift_z and the albedo's spatial contrast relative to the photograph); Appendix B / Table 4 gives it for every operating point together with the MIT ground-truth error (every operating point but the 10k NIID-Net checkpoint, which was not run on MIT).

  2. What it shows. Signed changes of medians under Eq. 5 versus Eq. 6: −28.5% versus −6.9% [−15.6%, +0.4%] for the C&A adapter, −38.5% versus −5.9% for the NIID-Net adapter, −39.9% versus +11.7% (worse than the base) for the λ=3 fine-tune, whose albedo keeps 0.54 of the base's spatial contrast. All 12 refinements of C&A have a higher mean MIT ground-truth error than the base (adapter +32.8%, λ=3 fine-tune +80.0%; NIID-Net adapter +111.3%), and the change in drift ratio and the change in error are rank-correlated at ρ = −0.79 (p = 0.002). At their headline operating points the consistency objectives tested lower the drift ratio without worsening full-IIW WHDR, mainly by flattening the albedo.

  3. Claims withdrawn or narrowed. "The invariance axis is improvable at no fidelity cost" is withdrawn (Note on this revision, abstract, §1, §2, contribution (4), §5, Limitation 4, conclusion, Fig. 1 caption). The abstract now states the ground-truth-error increase and limits the fine-tuning statement to λ=3 (λ=1: WHDR +66.2%). Finding 4 now says that the two-fold drift gap between C&A and Marigold is largely spatial contrast (0.50 vs. 0.99 of the photograph's; contrast-matched 0.407 vs. 0.391, overlapping intervals) and that model rankings depend on whether contrast is matched. The conclusion's recommendation is to report Eq. 5 with Eq. 6 and, where available, ground-truth error, and not to optimize invariance alone.

  4. Qualitative results (requested by zHrQ). Appendix D adds Figs. 5–9: every method of Table 1 and the three refined models on six scenes under two lights (per-scene drift and contrast-matched drift printed on the light-b panels); what the adapter and the fine-tune change (difference and residual maps); five MIT objects with ground truth and si-MSE; four IIW images with per-image WHDR. Images are chosen by stated rules. The previous Fig. 4 moved to the appendix.

  5. Related work. §2 states the Li & Snavely (2018) distinction with their section numbers (training loss, §5.2; evaluation WHDR/SAW/MIT, §6).

Unchanged from the first revision: research question, datasets, the Eq. 5 definition, every previously reported number, the controls (seed, ensemble, per-channel normalization, diffuse-only, HDR renders; the submitted linear-RGB-input control remains omitted, Table 7), the off-domain result (BigTime 0.595) and the unmeasured spectral axis (Limitation 2). Controls for the new quantity: identity scores exactly 1 under every normalization; the drift ratios recomputed from the cached predictions reproduce the archived per-scene values to within 7.6×10⁻⁵ (float16 cache storage), and the MIT recompute reproduces the archived C&A and adapter si-MSE to four decimals (Appendix B). The OpenReview abstract and TL;DR fields are updated to match the PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.