acceptodds
Under review as a conference paper at ICLR 2027

AskLight: Grounding Active-Light Reasoning for Multi-Illuminant White Balance

Abstract

Multi-illuminant white balance requires a spatially varying illumination estimate, but visible light sources poorly indicate the actual illumination: they may be switched off, miss sources outside the view, or share a region with other sources. We present AskLight, which connects semantic source reasoning to continuous illumination decomposition through an explicit source-to-region representation. A shared vision–language model (VLM) classifies detected proposals, then revises the active inventory by removing false actives, adding missed-visible or hidden hypotheses, and associating every retained hypothesis with a dominant illumination region. These relations initialize a variable-cardinality set of illuminant colors and spatial priors, which a recurrent decomposition backend refines into mixture weights and candidate chromaticities through differentiable physical composition. A central obstacle is that no real dataset provides per-source supervision of which sources are active, which are visible, and where each one casts light. We therefore introduce Infinigen-Light, a procedurally rendered multi-illuminant dataset with per-source annotations of activity, visibility, and region influence, which makes active-light reasoning both learnable and separately evaluable from final white balance accuracy. By grounding each illuminant candidate in an explicit source hypothesis rather than an unstructured slot or a global estimate, our method achieves the lowest mean illumination error on Infinigen-Light, LSMI, and MIIW, reducing the LSMI illumination error by 16.8% over the strongest baseline. We will release our source code.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-Illuminant White Balance, Vision-Language Models, Illumination Estimation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Yn71OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AskLight, a multi-illuminant white balance method in which a VLM classifies detected light-source proposals, revises the list of active sources, and associates each source with an illumination region. These source-to-region associations then initialize a network that estimates the per-pixel illumination and the corrected image. As there is no dataset with per-light-source information, the authors built Infinigen-Light, a procedurally rendered multi-illuminant dataset with per-source annotations of activity, visibility, and region-specific influence maps, which makes per-source reasoning possible. Their approach obtains the lowest mean illumination error on Infinigen-Light and on the real LSMI and MIIW datasets.

Critical strengths
  • The Infinigen-Light dataset, although synthetic, can be useful for the community.
  • AskLight outperforms all the compared methods in mean illumination error on Infinigen-Light and on all LSMI splits.
Critical weaknesses
  1. Motivation of the method. The authors propose AskLight, a method that uses a source-to-region representation as the interface between semantic reasoning and photometric estimation. My main concern is that the method is overkill. It only needs to estimate an illuminant for each pixel of the image, but it relies on a complex pipeline with billions of parameters (Grounding DINO, InternVL3-8B, and the recurrent network), and the authors do not justify this choice. At a bare minimum, they should report the total number of parameters, the peak VRAM usage, and the inference time.
  2. Relevance of the Infinigen-Light dataset. My second concern is whether the introduced dataset is useful. Infinigen scenes have a large domain gap with real scenes (odd textures, limited assets, scarce outdoor lighting), especially regarding light sources. Infinigen's procedural pipeline has only two types of light fixtures: ceiling lights and lamps (floor or table lamps). In Infinigen-Light, the sources are limited to these fixtures, windows, and an optional hidden source (Appendix A.1). Therefore, the VLM and the overall pipeline are trained (and, on Infinigen-Light, evaluated) on a very limited variety of light sources. The dataset is used to pre-train the VLM before adapting it to LSMI and MIIW, but the paper does not show how much this pre-training helps (e.g., by adapting the VLM to LSMI directly, without Infinigen-Light). I am worried that the limited variety of light sources hampers the generalization of the VLM.
  3. Does the VLM help? A table that disentangles the contribution of the VLM and its reasoning is missing. An ablation in which (a) N is fixed, (b) the ground-truth N (number of illuminants) and regions are given as an oracle, and (c) the VLM is replaced by a small non-VLM classifier with far fewer parameters would add value to the work.
  4. The active-light reasoning is barely used in real cases. Infinigen-Light images often contain several visible light sources. However, in the LSMI dataset, the second and third illuminants are rarely visible in the image, as they are usually placed behind the camera. On MIIW, every source is treated as hidden and every detected proposal is labeled as inactive. Therefore, the active-light reasoning, which is the core of the method, is mostly exercised on synthetic data.
Questions for authors
  1. The authors' AID reimplementation is ~1° worse than the published numbers. How did the authors reimplement the method, and what explains this gap?
  2. What is F1 in Table 2? The authors describe it as "correctly classified source states among all sources". Isn't that recall or accuracy?
  3. How do the output images look when the VLM fails? I would appreciate a discussion of the failure modes and their severity since, in white balance, we do not want a very bad image when a user captures a photo.
6 comments
AuthorsOpenReview ↗

Response(1/3)

Thank you for noting Infinigen-Light’s potential value and AskLight’s low mean illumination error. Your suggestion to compare the VLM with a lightweight classifier prompted us to test how much it contributes to the final white balance when the refinement network is held fixed.

W1. What does VLM cost? AskLight has 8.149B parameters, of which 7.968B belong to InternVL3-8B. Both VLM calls share the same weights. We benchmarked local inference on one 40 GB A100 after warm-up, using 50 images per dataset, batch size one, and CUDA synchronization. For GPT-6 Astra, we report time cost only.

Table 1. Computational Cost Comparison of AskLight and Other Methods

DatasetMethodParameters (B)Mean inference time (s/image)Peak allocated memory (GiB)Peak reserved memory (GiB)Mean compute (TFLOPs/image)
Infinigen-LightAskLight8.1495.1316.0116.7929.95
Infinigen-LightVLM-CC8.2926.8831.1232.8779.91
Infinigen-LightGPT-6 Astra—20.10*———
Infinigen-LightAID0.0072850.01740.3050.4280.103
Infinigen-LightLSMI-U0.0557780.01200.6270.6990.487
LSMI-AllAskLight8.1492.8816.1916.9732.54
LSMI-AllVLM-CC8.2927.0131.1232.8780.44
LSMI-AllGPT-6 Astra—16.26*———
LSMI-AllAID0.0072850.01700.3050.4280.103
LSMI-AllLSMI-U0.0557780.01260.6270.6990.487

* AskLight and VLM-CC timings cover the full pipeline on a local A100. For GPT-6 Astra, we used the API's server-side timestamps (completed_at - created_at) for the two source-reasoning tasks.

We recognize that AskLight is more computationally demanding than AID and LSMI-U. Nevertheless, it requires less inference time, GPU memory, and computation than VLM-CC. Its reported runtime is also shorter than GPT-6 Astra's. AskLight prioritizes interpretable, reasoning-based light-source-level decomposition and strong scene-lighting understanding rather than inference speed alone.

Table 2. Comparison of AskLight, GPT-6 Astra, and AID on Light-Source Understanding Tasks

ModelStatus Acc. (Task 1)Status F1 (Task 1)Src Acc. (Task 2)Img Acc.Per-region Count Acc.
GPT-6 Astra94.74%92.55%84.98%43.95%46.59%
AskLight (fine-tuned)94.50%89.01%92.69%69.73%63.60%
AID———23.54%—

On Infinigen-Light, fine-tuned AskLight leads zero-shot GPT-6 Astra across all three Task 2 metrics, despite the latter's higher Task 1 macro-F1. AskLight achieves 69.73% image-level count accuracy, compared with 43.95% for GPT-6 Astra and 23.54% for AID. Its additional advantages in source accuracy and per-region count accuracy further support its strong scene-lighting understanding.

W2. Does Infinigen-Light pretraining transfer to real images despite its limited fixture diversity? Infinigen-Light has two artificial fixture families. Varying the radius, thickness and curvature of ceiling lights yields 673 distinct geometries. Varying the stand, base and shade of floor lamps yields 75. Windows use four nominal color temperatures (4500, 5500, 6500 and 7500 K). Artificial-light colors are sampled independently in HSV space. Each of our synthetic datasets thus spans a wide range of fixture geometries and light colors.

To evaluate the benefit of Infinigen-Light pretraining on real scenes, we compared three training settings. Since AskLight fine-tunes the Infinigen-Light-pretrained VLM on LSMI for three epochs, we also trained an LSMI-only VLM for three epochs under the same fine-tuning budget. Additionally, we extended LSMI-only training to six epochs to examine whether longer training could close the gap.

We evaluated all three checkpoints on the same LSMI test images. Img Acc. is the percentage of images for which the total number of active light sources is correct. Per-region Count Acc. is the percentage of all regions for which the predicted active-source count is correct.

Table 3. Effect of synthetic pretraining on the 756 LSMI-All test images.

VLM trainingImg Acc. ↑Per-region Count Acc. ↑Count MAE ↓
No Infinigen-Light pretraining, 3 LSMI epochs50.13%74.37%0.499
No Infinigen-Light pretraining, 6 LSMI epochs51.59%76.29%0.487
Infinigen-Light pretrained + 3 LSMI epochs72.49%80.26%0.279
AuthorsOpenReview ↗

Response(2/3)

W3.Without VLM experiments. We compared five source inputs on Infinigen-Light. For the oracle upper bound, we used the ground-truth active-source count and regions with AskLight’s standard prior initialization during both training and testing. Because RepViT is a strong feature extractor, we repeated the comparison after replacing it with ResNet-18 as the downstream backbone. We kept each method’s source predictions fixed across backbones and trained each backend for 100 epochs. The ResNet-18 classifier in the last row replaces the VLM in predicting source states and regions.

Table 4. Mean angular errors on Infinigen-Light by different downstream backbone.

RepViTRepViTResNet-18ResNet-18
Source inputL Mean (°) ↓R Mean (°) ↓L Mean (°) ↓R Mean (°) ↓
AskLight3.9353.8884.2804.218
Gray-Pixel, fixed N=3+our stage D4.0383.9774.4364.367
Gray-Pixel, GT count+our stage D3.9563.8894.4094.346
Gray-Edge+our stage D4.8974.7924.8964.796
ResNet-18 classifier+our stage D4.0333.9774.4184.347
Oracle(GT count+regions)+our stage D3.5103.456——

Under RepViT, AskLight achieves nearly identical performance to Gray-Pixel with GT light-source count, demonstrating oracle-count-level downstream performance without ground-truth count information. Oracle region assignments further reduce the errors to 3.510° and 3.456°, highlighting the importance of accurate region-level light-source association. Moreover, AskLight's advantage over the strongest non-oracle baseline increases substantially with ResNet-18 (from 0.021° to 0.129° on L and from 0.001° to 0.128° on R), demonstrating the value of VLM-derived lighting information across lighter backbone.

We further evaluated how accurately each method counts active light sources in Infinigen-Light (Table 5), where AskLight achieves the best performance in both image-level and regional count accuracy.

Table 5. Active-source counts on Infinigen-Light. Higher is better.

Source inputImg Acc.Per-region Count Acc.
AskLight (ours)69.73%63.60%
ResNet-18 classifier43.27%56.10%
Gray-Edge, estimated count7.85%—

W4. Limited Real-World Validation of Visible Light-Source Activity Classification

Thank you for raising this point. We agree that our real-image experiments provide limited evidence for Task 1, which classifies each visible light-source proposal as active, inactive, or invalid. Across all 7,486 images in the original LSMI dataset, 3,397 (45.38%) were captured under one illuminant, 3,743 (50.00%) under two illuminants, and 346 (4.62%) under three illuminants. The official examples (https://www.dykim.me/projects/lsmi (samples)) show a variety of visible fixtures, but the annotations do not link individual fixtures to the illuminants they represent. We therefore cannot determine how often a contributing source is outside the frame or directly evaluate proposal-level status accuracy on LSMI. Infinigen-Light provides our most direct evaluation of this task.

The real-image evidence is strongest for Task 2. Even when a source is outside the frame, its effect on scene surfaces can help AskLight infer how many sources contribute and which region each illuminates most strongly. LSMI’s per-source contribution maps provide reference regions for this evaluation. On the same 756 real test images, with three epochs of LSMI fine-tuning in both settings, Infinigen-Light pretraining increased exact source-count accuracy from 50.13% to 72.49%. On MIIW, our protocol labels every detected proposal as inactive and treats all contributing sources as hidden. Under this protocol, AskLight achieved the lowest mean angular errors among the compared methods for both illumination estimation (L, 2.63°) and the white-balanced image (R, 2.82°).

We also tested whether end-to-end performance transfers to scenes with more lights than those seen during training. The training scenes contained 3–5 sources. In 50 additional rendered images, each scene contained 10 sources, with an average of 8.32 active sources per image. AskLight achieved mean L and R errors of 3.441° and 3.252°, compared with 4.167° and 3.908° for LSMI-U, the closest baseline on both metrics. This provides additional evidence that AskLight’s illumination estimates and white-balance corrections generalize to higher source counts within the synthetic setting.

Table 6:test an 50 10-light images

EvaluationMore Lights (50)More Lights (50)
MethodL (°)R (°)
AskLight3.4413.252
VLM-CC6.2115.801
LSMI-U4.1673.908
AIDSLOT5.5565.237
AuthorsOpenReview ↗

Response(3/3)

Q5. Why does our AID reimplementation differ from the published results? We reconstructed AID from the published architecture because the authors did not release the training code. The paper did not specify the optimizer or training duration, so we initially use the same training settings of our AskLight per dataset. We then retrained using the settings clarified by the authors on GitHub issues.

After updating the AID results, our method maintains strong overall performance across the evaluated datasets. It achieves the best results in all eight metrics on Infinigen-Light, seven out of eight metrics on LSMI-All, and both metrics on MIIW. On the Galaxy and Nikon subsets, our method also ranks first in four and five metrics, respectively. Although it does not achieve the best results on the Sony subset, it consistently ranks second in six out of eight metrics. We also update the new AID performance in our paper, please reference to table 1,4,8 and figure 4,7,8.

Our protocol masks color charts and uses a different image resolution 512×512 from AID’s 256 × 256. These differences may contribute to the remaining gap. Their individual effects have yet to be measured. An independent implementation reported 2.6° on the mixed-illuminant LSMI Galaxy subset. (https://github.com/DY112/AID/issues/4)

Q6. What does F1 measure in Table 2 of the paper? You are right that our wording defines accuracy. Table 2 reports macro-F1. We compute 2TP/(2TP + FP + FN) for each proposal state (active, inactive and invalid), then average the three scores equally. InternVL3-8B achieves 94.50% status accuracy and 89.01% macro-F1. We will correct the definition.

Q7 When VLM is wrong.

Thank you for raising this important point. We agree that the VLM is not always correct and may both overestimate and underestimate the number of active illuminants, particularly when light sources are occluded or visually ambiguous.

To address this concern, we have added a new subsection, “Behavior under Incorrect VLM Predictions,” together with representative qualitative examples from the Infinigen-Light, LSMI, and MIIW datasets (Fig. 9 and Page 23 of the revised manuscript).

As shown in these examples, although AskLight occasionally produces incorrect active-illuminant predictions, the downstream recurrent decomposition network can partially mitigate these errors by iteratively refining candidate chromaticities and spatial mixture weights. In particular, it can reduce the influence of falsely activated candidates while preserving much of the scene texture and recovering plausible illumination color patterns. However, this compensation is limited, especially when a true illuminant is incorrectly classified as inactive and excluded from the illumination mixture.

Reviewer_KC5XOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes AskLight for multi-illuminant white balance. A fine-tuned VLM decides which light sources are active and links each to an illumination region. Each source then initializes one illuminant candidate, refined by a RNN. The paper also introduces a synthetic dataset called Infinigen-Light, which has per-source activity, visibility and contribution annotations. The new pipeline improved over existing baselines.

Critical strengths
  1. Using the VLM only for discrete source reasoning, and leaving the color estimation to Gray Pixels and the backend, is a sensible design. It also naturally supports open-world reasoning and a variable number of illuminants.
  2. The pipeline behind the synthetic dataset Infinigen-Light is also valuable. It is used to generate the training data with labels at scale.
Critical weaknesses
  1. While it is natural to use VLM for the first stage reasoning, it is unclear whether an off-the-shelf VLM is already enough, e.g. Astra. A comparison with these zero-shot VLMs should be added.
  2. The generalization/extrapolation of the method is not well-tested. E.g., testing on scenes that contained a larger number of light sources than training data (maybe even stress test on the many-light rendering scenes), and also test on more diverse scene settings (e.g. move to outdoor in day and night, currently all test scenes are basically simple indoor scene combinations)
Questions for authors
  1. The VLM used in stage1 is fine-tuned. But it's unsure how much the fine-tuning has contributed. So a comparison with models without fine-tuning is needed.
  2. The distribution of lighting used in synthetic dataset is limited/too simple. I would suggest extending to at least 8 light sources in a single scene.
  3. It is unclear the robustness of stage D if stage C made mistakes. So I would like to see an added experiment that deliberately give stage D some errored input to see it robustness, and oracle input to see its upper bound.
3 comments

Thank you for recognizing the value of Infinigen-Light and our design choice to use the VLM for qualitative light-source reasoning and the downstream network for color estimation. We also appreciate your suggestions, which prompted us to examine how well AskLight generalizes to new lighting conditions and how robust our network estimates are to errors in the predicted sources.

W1 / Q1. What does fine-tuning contribute? We compared five off-the-shelf VLMs with our fine-tuned InternVL3-8B on test set of Infinigen-Light. Status Acc. and Status F1 measure visible-proposal classification, with F1 averaged over active, inactive, and invalid proposals. Src Acc. measures the fraction of ground-truth active sources recovered. Img Acc. and Per-region Count Acc. require an exact source count per image and per MIRF region, respectively.

Table 1. Zero-shot VLMs and trained AskLight on Infinigen-Light. Higher is better.

ModelStatus Acc. (Task 1)Status F1 (Task 1)Src Acc. (Task 2)Img Acc.Per-region Count Acc.
InternVL3-2B38.47%26.06%29.51%10.31%44.60%
Qwen2.5-VL-3B57.58%27.65%60.71%23.77%34.31%
Qwen2.5-VL-7B53.63%32.32%57.02%23.32%37.01%
InternVL3-8B47.80%35.16%38.51%13.90%51.66%
GPT-6 Astra94.74%92.55%84.98%43.95%46.59%
AskLight (fine-tuned)94.50%89.01%92.69%69.73%63.60%

Although GPT-6 Astra leads on both Task 1 metrics, our fine-tuned AskLight model performs best on all three final evaluation metrics, achieving 92.69% source accuracy, 69.73% exact image-level count accuracy, and 63.60% per-region count accuracy.

W2 / Q2. Transfer to more lights and to real scenes. Infinigen-Light training scenes contain three to five registered lights. We applied the original checkpoints trained on INF.LIGHT to 50 images with ten physical lights each, including two hidden active lights. We synthesized each image using the inf.light data pipeline. Forty-two images have eight active sources and eight have ten (mean 8.32).

In the indoor task, source contribution maps describe how the local illuminant changes across surfaces. Outdoors, sunlight and skylight mix differently in sun and shade. At night, localized lamps and low signal-to-noise ratios complicate color estimation.We initialized from LSMI weights and fine-tuned 10 epochs on each target domain. AskLight kept its VLM frozen and updated only the RepViT backend. The two datasets don’t give official train/test splits, so we set the Cube+ train/test split to 1364/342 and the NighttimeCC split to 410/103(80%/20%).

Table 2 reports the four methods on the same test images within each column. L and R are mean angular errors for illumination and white-balanced color, in degrees, averaged equally over images.

Table 2. Ten-light direct transfer and target-domain fine-tuning on Cube+ and Nighttime CC. Lower is better.

EvaluationMore Lights (50)More Lights (50)Cube+ (342)Cube+ (342)NighttimeCC (103)NighttimeCC (103)
MethodL (°)R (°)L (°)R (°)L (°)R (°)
AskLight3.4413.2521.4801.9083.0454.183
VLM-CC6.2115.8011.7692.2325.6207.709
LSMI-U4.1673.9083.3694.4753.2144.134
AIDSLOT5.5565.2371.6532.1483.3514.502

AskLight generalizes well to ten-light scenes and adapts well to Cube+ and Nighttime CC.

Q3. How robust is Stage D to upstream errors? We introduced VLM errors by changing 5%, 15%, or 25% of the Task 1 status labels predicted by the frozen Infinigen-Light VLM, rerunning Task 2, and reassigning the same fraction of sources to different valid regions. For the oracle upper bound, we used the ground-truth active-source count and regions with AskLight’s standard prior initialization during both training and testing. In all settings, we trained the downstream RepViT weight network for 100 epochs using the Stage D procedure described in the paper.

Table 3. Downstream errors with original, perturbed, and oracle source inputs on Infinigen-Light.

Stage C inputL Mean (°) ↓R Mean (°) ↓
AskLight3.9353.888
5% perturbed3.9593.888
15% perturbed4.0003.930
25% perturbed4.0144.045
Oracle (GT count + regions)3.5103.456

At 25% perturbation, L rises by 0.079° and R by 0.157°, showing that the weight network tolerates substantial source-prediction noise when trained with it. This gain of oracle results supports the value of identifying active sources and their regions. More accurate inputs improve the final white balance.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts in addressing my concerns, particularly the comparisons with off-the-shelf VLMs, the generalization experiments, and the robustness analysis.

These results have largely addressed my questions and strengthened the experimental validation. I am satisfied with the responses and will maintain my original positive score.

Thank you again for your thorough work.

We are truly encouraged by your positive feedback and sincerely appreciate your recognition of our additional efforts.

Your insightful questions motivated us to further investigate the capabilities of different VLMs in understanding complex lighting conditions and the generalization of AskLight across diverse illumination scenarios. Your comments on robustness and the oracle upper bound also helped us gain a deeper understanding of the scope of our method's capabilities.

Thank you again for your professional and valuable feedback, the time and effort you devoted to reviewing our paper, and your continued support throughout the review process.

Reviewer_qk5rOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AskLight for multi-illuminant white balance and introduces Infinigen-Light, a synthetic dataset with light-source activity, visibility, and contribution annotations. A frozen Grounding DINO detects possible visible sources, while Gray-Pixel and MIRF processing produces coarse illumination regions. A LoRA-tuned InternVL3-8B first labels detected sources as active, inactive, or invalid. It then adds missed or hidden source hypotheses and associates each retained source with an illumination region. These predictions initialize a variable number of illuminant colors and spatial weight maps. A recurrent network refines the colors and weights over three rounds to estimate the illumination field and corrected image. The VLM and recurrent backend are trained in separate stages. Results on Infinigen-Light, LSMI, and a reconstructed MIIW benchmark show competitive illumination and white-balance errors.

Critical strengths
  1. The problem motivation is meaningful. Visible light fixtures are not necessarily active, and off-screen sources may affect the image. Separating source validity, activity, and illuminated regions is a useful way to describe the ambiguity in multi-illuminant white balance.
  2. Infinigen-Light is a valuable part of the paper. Its source-level labels and scene-disjoint lighting-state splits allow the authors to supervise and evaluate active-source reasoning separately from the final white-balance output. The treatment of missed, inactive, and hidden sources is interesting.
  3. The division between semantic reasoning and numerical estimation is sensible. The VLM produces source hypotheses rather than camera-space RGB values, and the lightweight recurrent network predicts colors and spatial weights through a differentiable mixture model. This makes the overall method relatively easy to understand.
Critical weaknesses
  1. The specific benefit of VLM-based source reasoning is not isolated. Table 5 keeps the VLM-predicted source count in every setting, and changes only the Gray-Pixel or region initialization. The full method improves the illumination error from 4.01 to 3.94 degrees over the region-prior-only setting, but there is no matched backend using a purely photometric source inventory, a simple learned count predictor, or ground-truth counts and associations. Consequently, the main results do not establish how much of the improvement comes from the two-stage VLM rather than from the recurrent decomposition backend and the photometric priors.
  2. The claim that refined illuminant components remain grounded in physical sources is not fully supported. The VLM learns source-to-region labels on synthetic data, but the downstream backend is supervised only on the composed illumination field and corrected image, without candidate-to-source matching. Mixture decompositions are not unique: two candidates can represent the same source, or one candidate can absorb multiple sources, while the final illumination field remains accurate. The paper does not report per-source chromaticity or contribution-map accuracy after refinement, even though Infinigen-Light provides the labels needed to test this claim.
  3. Practical efficiency and some implementation details are missing. A single prediction uses Grounding DINO, Gray-Pixel/MIRF preprocessing, two sequential autoregressive calls to an 8B VLM, and three recurrent refinement rounds. Images and candidate refiners can be batched, but Task 2 depends on Task 1 and the refinement rounds are sequential. So, I think it is necessary to discuss the efficiency issue.
Questions for authors

Please refers to the weaknesses. I think this paper is an extension of VLM-CC, but the extention is reasonable and interesting. So, I keep a positive score.

4 comments
AuthorsOpenReview ↗

Response(1/2)

Thank you for highlighting why the distinction between source visibility and activity matters in practice: a visible fixture may be off, while a source outside the frame may still illuminate the scene. We especially appreciate your suggestions to evaluate individual source contributions and test the same refinement backend without the VLM. Both analyses helped us strengthen the paper.

W1. Contribution of VLM reasoning. VLM-CC’s use of qualitative judgments about residual color casts helped motivate our use of the VLM to infer which light sources are active and which regions they illuminate. Your comment prompted us to examine what the downstream network can achieve without VLM reasoning by training the same backend with alternative source inputs. To further assess the contribution of the VLM to downstream performance, we compared different light-source inputs under the same experimental settings. Given RepViT’s strong feature extraction capability, we also replaced it with ResNet-18 as the downstream backbone while keeping the light-source input fixed for each method. We trained each downstream network for 100 epochs using the same training settings reported in the paper. On the Infinigen-Light (INF.LIGHT) test set, we report the mean angular errors for illumination estimation (L) and white balance (R), in degrees. Lower values are better.

Table 1. Mean angular errors on Infinigen-Light by downstream backbone.

RepViTRepViTResNet-18ResNet-18
Source inputL Mean (°) ↓R Mean (°) ↓L Mean (°) ↓R Mean (°) ↓
AskLight3.9353.8884.2804.218
Gray-Pixel, fixed N=3+our stage D4.0383.9774.4364.367
Gray-Pixel, GT count+our stage D3.9563.8894.4094.346
Gray-Edge+our stage D4.8974.7924.8964.796
ResNet-18 classifier+our stage D4.0333.9774.4184.347
Oracle(GT count+regions)+our stage D3.5103.456——

Interestingly, AskLight achieves nearly identical performance to Gray-Pixel with GT light-source count (3.935° vs. 3.956° on L and 3.888° vs. 3.889° on R) with RepViT, suggesting that it can achieve oracle-count-level downstream performance without access to the ground-truth count. However, incorporating oracle region assignments further reduces the errors to 3.510° and 3.456°, highlighting the importance of accurately associating light sources with their corresponding image regions. Moreover, AskLight's advantage over the strongest baseline increases substantially with ResNet-18, from 0.021° to 0.129° on L and from 0.001° to 0.128° on R, demonstrating the value of VLM-derived lighting information across different backbones.

To directly assess how well each method understands scene lighting, Table 2 reports two measures of accuracy. Exact active-source count accuracy (Img Acc.) is the percentage of images for which the total number of active light sources is correct. Per-region Count Acc. is the percentage of all regions for which the predicted active-source count is correct. The experiment is performed on the test set of INF.LIGHT.

Table 2. Active-source counts on Infinigen-Light. Higher is better.

Source inputImg Acc.Per-region Count Acc.
AskLight (ours)69.73%63.60%
ResNet-18 classifier43.27%56.10%
Gray-Edge, estimated count7.85%—

AskLight outperforms the ResNet-18 source classifier on both measures, showing that the VLM more accurately identifies how many light sources are active and how they are distributed across regions.

W2. Accuracy of refined source contributions. We pair predicted and GT sources within each image by available source ID, then dominant region. For each pair we compute weight-map MSE over valid pixels. Extra predictions receive a zero GT map, and missed sources receive a zero predicted map. We average over all pairs within an image, then equally over images. The L error averages the angle between predicted and GT illumination vectors over valid, nonzero pixels, then over images. The weight-map MSE measures spatial contribution errors, including extra and missed sources.

Table 3. Composed illumination and source weight-map errors. Lower is better.

DatasetAIDSLOT L Mean ↓AskLight L Mean ↓AID MSE ↓AskLight MSE ↓
Infinigen-Light5.311°3.936°0.084290.07987
LSMI-All2.353°1.891°0.089350.15809
AuthorsOpenReview ↗

Response(2/2)

W3. Inference cost. AskLight has 8.149B unique parameters (detector 172.84M, VLM 7,967.70M, backend 7.99M). The two VLM calls are sequential and share weights, as do the three refinement rounds. We timed 50 images per dataset on one A100-SXM4-40GB after warm-up, using batch size one and CUDA synchronization. Times are seconds per image. Task 1 classifies visible-source status, and Task 2 recovers and assigns active sources to MIRF regions. Prior denotes candidate prior construction. Peak allocated / reserved denotes GPU memory in GiB.

Table 4. Inference time and peak GPU memory.

Inference stage / metricInfinigen-LightLSMI-All
Preview + detector0.150.15
Gray-Pixel / MIRF1.090.61
Task 11.730.86
Task 21.751.20
Prior0.390.04
Three rounds0.020.02
End-to-end mean5.132.88
Peak allocated / reserved (GiB)16.01 / 16.7916.19 / 16.97

Table 5 compares inference time for AskLight, VLM-CC, AID and LSMI-U on Infinigen-Light and LSMI-All, and reports parameter counts, peak GPU memory, and compute for AskLight and VLM-CC. For GPT-6 Astra, we report the time cost only. To further discuss the effect of our AskLight, Table 6 compares fine-tuned AskLight with zero-shot GPT-6 Astra on visible-source status classification, active-source identification, and active-source counts per image and per region.

Table 5. Computational Cost Comparison of AskLight and Other Methods:

DatasetMethodParameters (B)Mean inference time (s/image)Peak allocated memory (GiB)Peak reserved memory (GiB)Mean compute (TFLOPs/image)
Infinigen-LightAskLight8.1495.1316.0116.7929.95
Infinigen-LightVLM-CC8.2926.8831.1232.8779.91
Infinigen-LightGPT-6 Astra—20.10*———
Infinigen-LightAID0.0072850.01740.3050.4280.103
Infinigen-LightLSMI-U0.0557780.01200.6270.6990.487
LSMI-AllAskLight8.1492.8816.1916.9732.54
LSMI-AllVLM-CC8.2927.0131.1232.8780.44
LSMI-AllGPT-6 Astra—16.26*———
LSMI-AllAID0.0072850.01700.3050.4280.103
LSMI-AllLSMI-U0.0557780.01260.6270.6990.487
  • AskLight and VLM-CC timings cover the full pipeline on a local A100. For GPT-6 Astra, we used the API’s server-side timestamps (completed_at − created_at) for the two source-reasoning tasks.

Table 6.Comparison of AskLight, GPT-6 Astra, and AID on Light-Source Understanding Tasks

ModelStatus Acc. (Task 1)Status F1 (Task 1)Src Acc. (Task 2)Img Acc.Per-region Count Acc.
GPT-6 Astra94.74%92.55%84.98%43.95%46.59%
AskLight (fine-tuned)94.50%89.01%92.69%69.73%63.60%
AID———23.54%—

We acknowledge that AskLight incurs higher computational costs than lightweight methods such as AID and LSMI-U. However, it requires less inference time and computational resources than VLM-CC. More importantly, AskLight demonstrates strong scene-lighting understanding through interpretable, reasoning-based light-source-level decomposition.

Fine-tuned AskLight outperforms zero-shot GPT-6 Astra on all three Task 2 metrics (Table 6). The largest gain is in image-level accuracy, where AskLight reaches 69.73% versus GPT-6 Astra’s 43.95% and AID's 23.54%. AskLight also leads by 7.71 points in source accuracy and 17.01 points in per-region count accuracy against GPT-6 Astra.

Thanks for the author's response. I have no more questions. I still think this is a interesting paper to solve an challenge task in complex illumination. So, I keep my positive score.

We sincerely appreciate your continued support and encouraging feedback on our work. We are pleased that our additional experiments and explanations have addressed your concerns. Your constructive suggestions have helped us strengthen the experimental validation and better clarify the contributions of AskLight. Thank you again for your time, careful review, and constructive suggestions,

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank the Area Chair and all three reviewers for their careful assessment and constructive feedback. We appreciate the recognition of the motivation behind AskLight, the value of the Infinigen-Light dataset, and the design of separating semantic light-source reasoning from numerical illumination estimation. The reviews strengthened our experimental validation and clarified our methodological contributions.

The main concerns raised by the reviewers and our corresponding revisions are summarized below:

Reviewer qk5r – Contribution of VLM reasoning, source-level accuracy, and computational efficiency: We conducted additional experiments to isolate the contribution of VLM-based light-source reasoning. Specifically, we compared alternative source inputs, including Gray-Pixel, Gray-Edge, and a ResNet-18 classifier, while evaluating both RepViT and ResNet-18 as downstream backbones. We also introduced source-level evaluation metrics to assess the accuracy of refined illumination components, including illumination angular error and source contribution weight-map MSE. Furthermore, we benchmarked the computational cost of AskLight against VLM-CC and GPT-6 Astra, reporting parameter counts, inference time, GPU memory consumption, and computational requirements. The source-input comparisons and source-level evaluation have been incorporated into the revised manuscript (Tables 7 and 3, respectively).

Reviewer KC5X – Contribution of fine-tuning, generalization, and robustness: We compared fine-tuned AskLight with multiple off-the-shelf VLMs, including InternVL3 and Qwen2.5-VL variants, as well as zero-shot GPT-6 Astra, to evaluate the contribution of VLM fine-tuning. To assess generalization, we extended the evaluation to scenes containing up to ten light sources and conducted additional experiments on Cube+ and NighttimeCC, covering more diverse lighting conditions. We also introduced controlled perturbations to the upstream VLM predictions at different error levels (5%, 15%, and 25%) and an oracle experiment using ground-truth source counts and regions to evaluate the robustness and upper-bound performance of the downstream refinement network. The VLM backbone comparisons and ten-light generalization results have been incorporated into the supplementary material (Tables 9 and 11, respectively).

Reviewer Yn71 – Synthetic pretraining, real-world validation, methodological contribution, and failure analysis: We further investigated the contribution of Infinigen-Light pretraining by comparing LSMI-only training with different training epochs against synthetic pretraining followed by LSMI fine-tuning. We also expanded the ablation study by comparing VLM-based source reasoning with alternative non-VLM source inputs and a lightweight ResNet-18 classifier, together with oracle source information. Regarding real-world validation, we clarified the limitations of evaluating visible-source activity classification on LSMI and MIIW, and provided additional evidence of generalization through source-count evaluation and downstream white-balance results on real datasets. In addition, we updated the AID reimplementation results using the training settings clarified by the original authors, corrected the definition of macro-F1, and added a new subsection, "Behavior under Incorrect VLM Predictions," with qualitative examples to analyze the robustness and limitations of AskLight when the VLM produces incorrect predictions. To further demonstrate AskLight's potential applications in lighting engineering beyond color correction, we highlighted its inherent capabilities in visible light-source activity detection and hidden light-source inference, and further evaluated two lighting-state monitoring tasks: active-source counting and light-switch direction recognition. We also evaluated generalization on three challenging datasets (Nighttime CC, CUBE+, and PolyU Pure Color V1) and explored lightweight VLM variants with MLP readouts, achieving 18.4–22.2× faster end-to-end inference while maintaining strong color correction performance. The synthetic pretraining ablation, alternative source-input comparisons, real-world source-recovery evaluation, updated AID results, and failure analysis have all been incorporated into the revised manuscript and supplementary material (Tables 1,4,7,8 and figures 4,7,8, Appendix Section D.3 with Figure 9).

Overall, these revisions clarify VLM reasoning, validate source-level predictions, test generalization under challenging lighting, and analyze computational efficiency and failure cases. The major experimental findings and corresponding analyses have been incorporated into the revised manuscript and supplementary material, while additional results are provided in the rebuttal.

We sincerely appreciate the reviewers' insightful comments and suggestions, which have helped us improve the clarity, completeness, and scientific rigor of this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.