- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents AskLight, a multi-illuminant white balance method in which a VLM classifies detected light-source proposals, revises the list of active sources, and associates each source with an illumination region. These source-to-region associations then initialize a network that estimates the per-pixel illumination and the corrected image. As there is no dataset with per-light-source information, the authors built Infinigen-Light, a procedurally rendered multi-illuminant dataset with per-source annotations of activity, visibility, and region-specific influence maps, which makes per-source reasoning possible. Their approach obtains the lowest mean illumination error on Infinigen-Light and on the real LSMI and MIIW datasets.
Critical strengths
- The Infinigen-Light dataset, although synthetic, can be useful for the community.
- AskLight outperforms all the compared methods in mean illumination error on Infinigen-Light and on all LSMI splits.
Critical weaknesses
- Motivation of the method. The authors propose AskLight, a method that uses a source-to-region representation as the interface between semantic reasoning and photometric estimation. My main concern is that the method is overkill. It only needs to estimate an illuminant for each pixel of the image, but it relies on a complex pipeline with billions of parameters (Grounding DINO, InternVL3-8B, and the recurrent network), and the authors do not justify this choice. At a bare minimum, they should report the total number of parameters, the peak VRAM usage, and the inference time.
- Relevance of the Infinigen-Light dataset. My second concern is whether the introduced dataset is useful. Infinigen scenes have a large domain gap with real scenes (odd textures, limited assets, scarce outdoor lighting), especially regarding light sources. Infinigen's procedural pipeline has only two types of light fixtures: ceiling lights and lamps (floor or table lamps). In Infinigen-Light, the sources are limited to these fixtures, windows, and an optional hidden source (Appendix A.1). Therefore, the VLM and the overall pipeline are trained (and, on Infinigen-Light, evaluated) on a very limited variety of light sources. The dataset is used to pre-train the VLM before adapting it to LSMI and MIIW, but the paper does not show how much this pre-training helps (e.g., by adapting the VLM to LSMI directly, without Infinigen-Light). I am worried that the limited variety of light sources hampers the generalization of the VLM.
- Does the VLM help? A table that disentangles the contribution of the VLM and its reasoning is missing. An ablation in which (a) N is fixed, (b) the ground-truth N (number of illuminants) and regions are given as an oracle, and (c) the VLM is replaced by a small non-VLM classifier with far fewer parameters would add value to the work.
- The active-light reasoning is barely used in real cases. Infinigen-Light images often contain several visible light sources. However, in the LSMI dataset, the second and third illuminants are rarely visible in the image, as they are usually placed behind the camera. On MIIW, every source is treated as hidden and every detected proposal is labeled as inactive. Therefore, the active-light reasoning, which is the core of the method, is mostly exercised on synthetic data.
Questions for authors
- The authors' AID reimplementation is ~1° worse than the published numbers. How did the authors reimplement the method, and what explains this gap?
- What is F1 in Table 2? The authors describe it as "correctly classified source states among all sources". Isn't that recall or accuracy?
- How do the output images look when the VLM fails? I would appreciate a discussion of the failure modes and their severity since, in white balance, we do not want a very bad image when a user captures a photo.
6 comments
Response(2/3)
W3.Without VLM experiments. We compared five source inputs on Infinigen-Light. For the oracle upper bound, we used the ground-truth active-source count and regions with AskLight’s standard prior initialization during both training and testing. Because RepViT is a strong feature extractor, we repeated the comparison after replacing it with ResNet-18 as the downstream backbone. We kept each method’s source predictions fixed across backbones and trained each backend for 100 epochs. The ResNet-18 classifier in the last row replaces the VLM in predicting source states and regions.
Table 4. Mean angular errors on Infinigen-Light by different downstream backbone.
| RepViT | RepViT | ResNet-18 | ResNet-18 | |
|---|---|---|---|---|
| Source input | L Mean (°) ↓ | R Mean (°) ↓ | L Mean (°) ↓ | R Mean (°) ↓ |
| AskLight | 3.935 | 3.888 | 4.280 | 4.218 |
| Gray-Pixel, fixed N=3+our stage D | 4.038 | 3.977 | 4.436 | 4.367 |
| Gray-Pixel, GT count+our stage D | 3.956 | 3.889 | 4.409 | 4.346 |
| Gray-Edge+our stage D | 4.897 | 4.792 | 4.896 | 4.796 |
| ResNet-18 classifier+our stage D | 4.033 | 3.977 | 4.418 | 4.347 |
| Oracle(GT count+regions)+our stage D | 3.510 | 3.456 | — | — |
Under RepViT, AskLight achieves nearly identical performance to Gray-Pixel with GT light-source count, demonstrating oracle-count-level downstream performance without ground-truth count information. Oracle region assignments further reduce the errors to 3.510° and 3.456°, highlighting the importance of accurate region-level light-source association. Moreover, AskLight's advantage over the strongest non-oracle baseline increases substantially with ResNet-18 (from 0.021° to 0.129° on L and from 0.001° to 0.128° on R), demonstrating the value of VLM-derived lighting information across lighter backbone.
We further evaluated how accurately each method counts active light sources in Infinigen-Light (Table 5), where AskLight achieves the best performance in both image-level and regional count accuracy.
Table 5. Active-source counts on Infinigen-Light. Higher is better.
| Source input | Img Acc. | Per-region Count Acc. |
|---|---|---|
| AskLight (ours) | 69.73% | 63.60% |
| ResNet-18 classifier | 43.27% | 56.10% |
| Gray-Edge, estimated count | 7.85% | — |
W4. Limited Real-World Validation of Visible Light-Source Activity Classification
Thank you for raising this point. We agree that our real-image experiments provide limited evidence for Task 1, which classifies each visible light-source proposal as active, inactive, or invalid. Across all 7,486 images in the original LSMI dataset, 3,397 (45.38%) were captured under one illuminant, 3,743 (50.00%) under two illuminants, and 346 (4.62%) under three illuminants. The official examples (https://www.dykim.me/projects/lsmi (samples)) show a variety of visible fixtures, but the annotations do not link individual fixtures to the illuminants they represent. We therefore cannot determine how often a contributing source is outside the frame or directly evaluate proposal-level status accuracy on LSMI. Infinigen-Light provides our most direct evaluation of this task.
The real-image evidence is strongest for Task 2. Even when a source is outside the frame, its effect on scene surfaces can help AskLight infer how many sources contribute and which region each illuminates most strongly. LSMI’s per-source contribution maps provide reference regions for this evaluation. On the same 756 real test images, with three epochs of LSMI fine-tuning in both settings, Infinigen-Light pretraining increased exact source-count accuracy from 50.13% to 72.49%. On MIIW, our protocol labels every detected proposal as inactive and treats all contributing sources as hidden. Under this protocol, AskLight achieved the lowest mean angular errors among the compared methods for both illumination estimation (L, 2.63°) and the white-balanced image (R, 2.82°).
We also tested whether end-to-end performance transfers to scenes with more lights than those seen during training. The training scenes contained 3–5 sources. In 50 additional rendered images, each scene contained 10 sources, with an average of 8.32 active sources per image. AskLight achieved mean L and R errors of 3.441° and 3.252°, compared with 4.167° and 3.908° for LSMI-U, the closest baseline on both metrics. This provides additional evidence that AskLight’s illumination estimates and white-balance corrections generalize to higher source counts within the synthetic setting.
Table 6:test an 50 10-light images
| Evaluation | More Lights (50) | More Lights (50) |
|---|---|---|
| Method | L (°) | R (°) |
| AskLight | 3.441 | 3.252 |
| VLM-CC | 6.211 | 5.801 |
| LSMI-U | 4.167 | 3.908 |
| AIDSLOT | 5.556 | 5.237 |
Response(3/3)
Q5. Why does our AID reimplementation differ from the published results? We reconstructed AID from the published architecture because the authors did not release the training code. The paper did not specify the optimizer or training duration, so we initially use the same training settings of our AskLight per dataset. We then retrained using the settings clarified by the authors on GitHub issues.
After updating the AID results, our method maintains strong overall performance across the evaluated datasets. It achieves the best results in all eight metrics on Infinigen-Light, seven out of eight metrics on LSMI-All, and both metrics on MIIW. On the Galaxy and Nikon subsets, our method also ranks first in four and five metrics, respectively. Although it does not achieve the best results on the Sony subset, it consistently ranks second in six out of eight metrics. We also update the new AID performance in our paper, please reference to table 1,4,8 and figure 4,7,8.
Our protocol masks color charts and uses a different image resolution 512×512 from AID’s 256 × 256. These differences may contribute to the remaining gap. Their individual effects have yet to be measured. An independent implementation reported 2.6° on the mixed-illuminant LSMI Galaxy subset. (https://github.com/DY112/AID/issues/4)
Q6. What does F1 measure in Table 2 of the paper? You are right that our wording defines accuracy. Table 2 reports macro-F1. We compute 2TP/(2TP + FP + FN) for each proposal state (active, inactive and invalid), then average the three scores equally. InternVL3-8B achieves 94.50% status accuracy and 89.01% macro-F1. We will correct the definition.
Q7 When VLM is wrong.
Thank you for raising this important point. We agree that the VLM is not always correct and may both overestimate and underestimate the number of active illuminants, particularly when light sources are occluded or visually ambiguous.
To address this concern, we have added a new subsection, “Behavior under Incorrect VLM Predictions,” together with representative qualitative examples from the Infinigen-Light, LSMI, and MIIW datasets (Fig. 9 and Page 23 of the revised manuscript).
As shown in these examples, although AskLight occasionally produces incorrect active-illuminant predictions, the downstream recurrent decomposition network can partially mitigate these errors by iteratively refining candidate chromaticities and spatial mixture weights. In particular, it can reduce the influence of falsely activated candidates while preserving much of the scene texture and recovering plausible illumination color patterns. However, this compensation is limited, especially when a true illuminant is incorrectly classified as inactive and excluded from the illumination mixture.
Response(1/3)
Thank you for noting Infinigen-Light’s potential value and AskLight’s low mean illumination error. Your suggestion to compare the VLM with a lightweight classifier prompted us to test how much it contributes to the final white balance when the refinement network is held fixed.
W1. What does VLM cost? AskLight has 8.149B parameters, of which 7.968B belong to InternVL3-8B. Both VLM calls share the same weights. We benchmarked local inference on one 40 GB A100 after warm-up, using 50 images per dataset, batch size one, and CUDA synchronization. For GPT-6 Astra, we report time cost only.
Table 1. Computational Cost Comparison of AskLight and Other Methods
* AskLight and VLM-CC timings cover the full pipeline on a local A100. For GPT-6 Astra, we used the API's server-side timestamps (
completed_at - created_at) for the two source-reasoning tasks.We recognize that AskLight is more computationally demanding than AID and LSMI-U. Nevertheless, it requires less inference time, GPU memory, and computation than VLM-CC. Its reported runtime is also shorter than GPT-6 Astra's. AskLight prioritizes interpretable, reasoning-based light-source-level decomposition and strong scene-lighting understanding rather than inference speed alone.
Table 2. Comparison of AskLight, GPT-6 Astra, and AID on Light-Source Understanding Tasks
On Infinigen-Light, fine-tuned AskLight leads zero-shot GPT-6 Astra across all three Task 2 metrics, despite the latter's higher Task 1 macro-F1. AskLight achieves 69.73% image-level count accuracy, compared with 43.95% for GPT-6 Astra and 23.54% for AID. Its additional advantages in source accuracy and per-region count accuracy further support its strong scene-lighting understanding.
W2. Does Infinigen-Light pretraining transfer to real images despite its limited fixture diversity? Infinigen-Light has two artificial fixture families. Varying the radius, thickness and curvature of ceiling lights yields 673 distinct geometries. Varying the stand, base and shade of floor lamps yields 75. Windows use four nominal color temperatures (4500, 5500, 6500 and 7500 K). Artificial-light colors are sampled independently in HSV space. Each of our synthetic datasets thus spans a wide range of fixture geometries and light colors.
To evaluate the benefit of Infinigen-Light pretraining on real scenes, we compared three training settings. Since AskLight fine-tunes the Infinigen-Light-pretrained VLM on LSMI for three epochs, we also trained an LSMI-only VLM for three epochs under the same fine-tuning budget. Additionally, we extended LSMI-only training to six epochs to examine whether longer training could close the gap.
We evaluated all three checkpoints on the same LSMI test images. Img Acc. is the percentage of images for which the total number of active light sources is correct. Per-region Count Acc. is the percentage of all regions for which the predicted active-source count is correct.
Table 3. Effect of synthetic pretraining on the 756 LSMI-All test images.