acceptodds
Withdrawn from ICLR 2027

Lightmap-GS: Neural Compression of Dynamic Lightmaps via 2D Gaussians

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Dynamic HDR lightmaps store changing illumination on fixed, sparsely occupied UV atlases. Their compression requires spatial capacity to follow atlas detail while sharing information across lighting states. We present \method, a content-specific representation that couples fixed anisotropic 2D Gaussian kernels with a factorized temporal color field. A multiresolution TriPlane decoder predicts color residuals, and tile-local normalized Top- reconstruction supports full-frame decoding and direct texel evaluation. Capacity-controlled ablations show gains of 2.17 dB from factorized features, 1.30 dB from residual color prediction, and 1.27 dB from adaptive allocation in the tested configurations. On 40 synthetic lightmap sequences, \method achieves 35.41 dB under a shared-scale evaluation protocol. A twelve-state, sequence occupies approximately 0.41 MiB versus 72 MiB in Float32, with 1.96 ms median full-frame reconstruction on an RTX 4090. Additional lighting states, external assets, and an Unreal Engine integration characterize temporal coverage and deployment costs. A matched-storage diagnostic improves the weakest fitted-state quality over temporal Gaussian alternatives, while higher-precision HEVC achieves substantially better full-atlas fidelity. Optimized neural and hardware-video comparisons further motivate evaluating selective texel access separately from bulk reconstruction and rate–distortion performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
2DGS, Image Compression, Lightmap

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8AvzOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a compact representation for time-varying HDR lightmaps, i.e., baked illumination in a UV atlas under multiple lighting states. The method represents the atlas with a fixed set of learned 2D Gaussians shared across all states following Image-GS, and predicts their time-dependent colors with a TriPlane feature field, so that only appearance changes across states and individual time can be queried locally. The evaluation is done on 40 Blender-generated scenes across 11 hours, comparing rendering quality, size, and efficiency, showing the proposed method has better rendering, smaller size, and more efficient. Ablation studies show each component is helpful.

Critical strengths
  1. The paper introduces a method that outperforms all listed existing methods, with smaller size and faster speed.
  2. Ablation studies show that each proposed component is helpful.
Critical weaknesses
  1. The paper is hard to understand for me. There is not teaser figure, and the method figure is not informative and low resolution. The writing is not very intuitive, and I failed to understand why the task is important, what is the required properties to have for the task, and why is the proposed method a good fit. It mostly just introduce what has been done.
  2. The introduction to existing methods and distinguishment between proposed components and them are unclear. If I understand correctly, the method is combining the image-DS with triplane time dimension triplane, but this is not clearly stated and differentiated.
  3. Based on the ablation study, even the worst ablation has 33.31 dB, while existing methods typically range around 26 - 28 dB. Why is this? Is the base model just better? If none of the proposed component is added, what's the base performance?
Questions for authors

As listed in critical weakness.

Reviewer_6d4UOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Lightmap-GS, a content-specific neural representation for compressing dynamic HDR lightmaps. The method uses fixed anisotropic 2D Gaussian kernels as spatial basis functions shared across lighting states, with a factorized TriPlane network predicting time-dependent color residuals at kernel centers. The approach enables both full-frame reconstruction and direct texel evaluation through tile-local normalized Top-K blending. The authors evaluate their method on 40 synthetic lightmap sequences with twelve states each, demonstrating substantial compression (0.41 MiB vs 72 MiB raw) with competitive reconstruction quality. They provide capacity-controlled ablations isolating the contributions of factorized features, residual prediction, and adaptive spatial allocation, as well as comparisons against Gaussian baselines, optimized neural alternatives (Instant-NGP, Temporal-NTC), and hardware video codecs (HEVC/H.264).

Critical strengths

1.The paper accurately identifies the unique challenges in dynamic lightmap compression, including irregular UV occupancy, fixed geometry, and temporal illumination changes. Furthermore, the proposed separated design renders the representation both highly efficient and intuitive. 2.The ablation study is comprehensive and sufficiently validates the effectiveness of each proposed component. 3.The evaluation is thorough and multi-dimensional, assessing the method across reconstruction quality, storage footprint, decoding speed, per-texel query capability, and scalability with an increasing number of lighting states.

Critical weaknesses

1.Limited general advantage over existing alternatives: Storage-matched experiments reveal that optimized neural baselines (e.g., Temporal-NTC* H achieves 46.12 dB vs. Lightmap-GS's 36.38 dB across three scenes) and hardware video codecs (e.g., HEVC achieves 65.02 dB at 0.5 MiB) substantially outperform the proposed method in terms of full-atlas fidelity. Although the paper acknowledges this limitation, the practical applicability and specific use cases where the proposed method holds a distinct advantage remain narrow.

2.Restricted scope of capacity-matched comparisons: The most rigorous storage-controlled evaluation (Table 2) is limited to a single scene. This narrow scope restricts confidence in the generalization of the observed per-state fidelity benefits across diverse environments.

3.Exclusive reliance on synthetic evaluation (Minor): Although the inclusion of external Blender assets diversifies the test set, all evaluations are conducted exclusively on synthetic data featuring relatively smooth illumination transitions. Consequently, the method's robustness on real-world lightmap sequences characterized by complex, dynamic shadow boundaries remains unverified.

4.The ablation study on the scale head reveals a significant performance degradation. This suggests that the optimization procedure may be highly sensitive to parameter initialization and learning rate scheduling. Would it be possible to elaborate on the causes of this observation?

Questions for authors

Please refer to Weaknesses.

Reviewer_Qu3oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes using Gaussian splatting to represent high dynamic range time-varying light maps. The authors train a fixed number (10k) of Gaussians to represent a 512x1024 light map with S states (primarily tested at 12) that represent a series of times. The spatial parameters of the Gaussians are fixed across the time sequence. The system uses a single network to predict color residuals given the time and UV coordinates of a Gaussian. The color residuals modify a learned canonical color for the light map. At inference time, the Gaussians can be used to predict the entire map or individual pixels, as needed. To predict color at a given point, the system identifies the K (10) closest Gaussians, uses the network and canonical color to estimate the actual color of each Gaussian, and then blends the results to get the particular pixel color. The system uses a grid to limit the number of Gaussians evaluated to get the closest K.

The paper includes an evaluation of the system against other Gaussian based methodologies (updated by the authors to work on light maps) using a data set of 40 sequences. The paper also includes an evaluation against other learned methods using a data set of three scenes and a qualitative and quantitative evaluation at two different times on one scene. There are also multiple studies looking at timing, latency, and memory usage using three or five scenes.

The primary contribution of the paper is the design of the learned representation: fixed spatial Gaussians with a single lightweight network to learn the time-varying color residuals.

Critical strengths

The overall concept of using fixed Gaussians and a single network to predict residuals is interesting. It is a conceptual mix of Gaussian Splatting and NeRF that enables a reasonably fast and compact representation of HDR light maps.

Critical weaknesses

The primary weakness is the extent and detail of the evaluation. Using 40 light maps is potentially acceptable for a preliminary evaluation. But only the other Gaussian methodologies are compared using the whole data set. The paper presents results versus other learned methodologies using only three sequences (table 3) or only one sequence (table 2 and figure 2). The results versus other learned methods in table 3 do not show improvement. The qualitative results shown in figure 2 raise concerns about the spatial distribution of the Gaussians, as fine details of both the lighter and darker areas are missing, though the other methods are able to capture them.

There are a number of minor issues with the paper, including the writing. There are some parts of the paper that are clear and well-written, while other parts need significant editing.

The following are more specific comments.

Abstract: Please have someone not familiar with the paper review the abstract.

Q1: Why do we need dynamic HDR lightmaps? What is their utility? Please start the abstract with some kind of motivation for the work. HDR is pretty common, but all acronyms should be defined with the first use.

Q2: Regarding the sentence: "Additional lighting states, external assets, and an Unreal Engine integration characterize temporal coverage and deployment costs." How do lighting states, assets, or an unreal engine characterize something? They can represent deployment costs, or be deployment costs, but they aren't agents that do things. What does temporal coverage mean in this context? Note that 'Unreal Engine' is not mentioned in the entire paper except in the abstract. Why is it there?

Q3: What is a 'matched-storage diagnostic' or 'weakest fitted-state quality'?

Q4: HEVC is not defined. But if HEVC does better, what is the purpose of this paper? The sentence says HEVC does better than Gaussian alternatives, but the paper is about using Gaussians. What is this supposed to tell the reader?

The abstract is the only thing most people will ever read about your paper. Explaining the problem you are solving, the key novel elements, and a short summary of how well it performs are the key things that need to come across. Including the ablation study results in the abstract is pretty much meaningless to anyone who hasn't already read the paper. I'd strongly recommend re-writing the abstract to provide more meaning and less detail. For example, the sentence

"A twelve-state, 512 ×1024 sequence occupies approximately 0.41 MiB versus 72 MiB in Float32, with 1.96 ms median full-frame reconstruction on an RTX 4090."

could be re-written as

"On a twelve-state, 512 ×1024 sequence, the proposed method achieves 175:1 memory compression with <2ms median full-frame reconstruction on an RTX 4090."

This avoids boring your reader with too many numbers and clarifies the impact of the method. Focus the abstract on meaning and impact.

The start of the introduction is much better.

One of the consequences of using fixed spatial Gaussians is that the kernel distribution and size must reflect the needs of the entire lighting sequence. That implies that in a lighting sequence with moving high frequency lighting, there would need to be a large number of almost uniformly small kernels. The paper uses a fixed number of kernels, which suggests the results may be at risk of missing high frequency information.

Lines 95:98 have one sentence with a colon and the next sentence with a semi-colon. While possibly grammatically correct, these patterns of writing create an uneven reading experience that does not help with understanding. The semi-colon in this example is gratuitous and could be replaced with 'so'.

Please use commas or conjunctions to connect phrases or else create separate sentences that flow together rather than asking the reader to continuously move down and up hierarchies of concepts or mentally insert connectors that should be there. There are a number of other examples, such as line 105, the caption of figure 1, the caption of table 1 (twice), the capture of figure 2 (twice), line 140, and line 143, and many others, that read like listening to someone who is always going off on tangents for half a sentence. Most of the text is much, much better than that and much easier to parse. If you want a person to read and understand the paper, please avoid repeated use of colons and semi-colons when they are not necessary.

Line 106-107: 'This inverse scaling does not make lossy reconstruction lossless.' reads like a throwaway sentence with no explanation. I assume this is because the representation uses a fixed number of bits per position, so there is always quantization in the reconstruction.

line 184: why log-domain reconstruction? It makes sense to me, but what is the reasoning behind it (or is there a citation?).

Lines 197-200: It would be helpful to scan through the text and avoid changing from active to passive voice in alternating sentences. Lines 197-200 start in active voice, smashes two different thoughts together with a semi-colon, moves from "We report" before the semi-colon, to "match-cap study instead counts" after the semi-colon, to "objective is assumed." in the final sentence. Active voice is a bit easier to read, but please choose one or the other.

Line 215: why use MiB, and not just the typical MB (megabyte)? It makes me think you're using some other unit.

Line 250: what does the following mean? "; we do not combine that rounded size with an inconsistent 0.53 bpp claim."

Line 251: 'denoted by asterisks." where? I assume this is in table 2, which is not referenced until several paragraphs later.

Line 253: 'three locally available scenes' what does 'locally' mean in this context?

Lines 252-257 :It's unclear what the paragraph 252-257 is describing because there is no reference to a table, a figure, or particular experiments. This paragraph seems to be important because it is describing the source of the 40-scene test set (author-adaptations, how, what adaptations?), that only five of the 40 scenes are providing BC6H (an acronym never identified in the text), and that some of the experiments use only three scenes. Some of these are clarified later in the paper, but reading the paper linearly results in confusion.

Line 297: 'The fixed 6:00 row illustrates' assume this is in Figure 2?

Results section:

A scene 17 result of 30.03 dB is reported in the appendix (assume for Lightmap-GS). It's referenced on line 298 of the paper without context. 'These fits do not explain' is meaningless. What fits? What is a fit? Why is one number being reported in the appendix and referenced in the paper, but a different number is used in table 2?

Table 5 caption: Please remove the semi-colon and insert 'with' or 'using'.

Lines 352-354: remove the semi-colons. Commas are appropriate and cleaner. More importantly, the connections between the ablation study table and the three clauses are not clear. For example, 'canoncial colors make the residual task easier' does not really match up with the meaning of 'Direct temporal color instead of residual'. The text in line 352 says using 'canonical colors' make things easier, but the table indicates that using residuals instead of temporal colors makes things easier. The other two are ok, if you can connect TriPlane to 'factorized features' and connect 'uniformly filling valid charts' to 'Uniform valid-region'. Why not use the same language in both places?

What is the overall message of the latency timing studies? The Lightmap-GS system has a low latency for small numbers of queries, but much higher for large numbers. Which tends to occur in practice? Is the latency for 65k queries fast enough?

Table 8 shows decreasing performance with longer temporal sequences. The paragraph in lines 416-424 duplicates the contents of table 8 but doesn't make the purpose of the text or the meaning of the results clear. Given earlier commentary, using 24 or 48 states includes 'lights off' time. Does that mean having to represent dark scenes is more challenging for the residual color network to learn, or is it the strong contrast between light and dark which may skew the spatial allocation of the Gaussians? Removing the existing paragraph (which just duplicates information from the table) and replacing it with an analysis or more enlightening summary would help. Understanding why the performance is decreasing is important.

Spatial Distribution of the Gaussians:

The primary learning representation in this paper is the distribution and training of spatial Gaussians and the globally learned color features. The spacial allocation is done in stages, with 50% generated at the beginning and 12.5% every 500 steps. The system uses a fixed number of kernels (10k).

As this is the critical innovation, the details on spatial placement need to be clear.

The initial set uses 'gradient-weighted placement with random exploration'.

  • Gradients of what?
  • Random exploration makes it sound like a process. Are they placed using a uniform random distribution, a Poisson distribution, jitter sampling, or using some other sampling process?
  • 'Subsequent additions use reconstruction error to focus capacity'. Does it use reconstruction error as a deterministic placement metric (e.g. local peaks) or is the reconstruction error used as a pdf for sampling new locations?

Section 3.4 should include these types of details given the centrality of the Gaussian placements. There are no ablation studies on placement methods.

Given the placement loss function is L1, the losses are a simple average over all states. If one state has a substantially different spatial distribution, the L-1 loss penalizes the average, not the max or the squared errors. Therefore, a large error E in one state is equal to losses of size E/N across N states, which discourages spatial adaptation to cover outliers in contrast to an L-2 or L-inf loss metric. There is no ablation study of the loss error metric.

As noted by the authors, the generality of the proposed method is also limited by the size of the test set. Many of the results use only five, or even just one scene. Scene 17 is chosen by the authors for evaluating the results across states, but there is no discussion of whether Scene 17 is unique, typical, easy, or hard. Does it have properties that make it challenging?

Looking at Figure 2, Lightmap-GS struggles with fine details, both positive (small lit areas that are missing) and negative (small areas of darkness withinin lighter patches). These suggest the Gaussians are either not being placed well or that there are not enough of them to properly sample the higher frequency elements of the scene. This ties in with the concerns about how the Gaussians are being placed and the loss function being used to learn their positions and variances.

Questions for authors

How are the Gaussians distributed initially and in the subsequent stages?

Why was sequence 17 chosen for several of the evaluations? Is it representative, challenging, easy, or does it contain elements that highlight something about the results?

The proposed method seems to miss a number of high frequency details in the qualitative results on scene 17. Why? Are there not enough Gaussians to accurately represent them, or are the Gaussians moving to other locations due to the loss function specifics?

Given table 8, what is the value of using the Lightmap-GS methodology?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.