The primary weakness is the extent and detail of the evaluation. Using 40 light maps is potentially acceptable for a preliminary evaluation. But only the other Gaussian methodologies are compared using the whole data set. The paper presents results versus other learned methodologies using only three sequences (table 3) or only one sequence (table 2 and figure 2). The results versus other learned methods in table 3 do not show improvement. The qualitative results shown in figure 2 raise concerns about the spatial distribution of the Gaussians, as fine details of both the lighter and darker areas are missing, though the other methods are able to capture them.
There are a number of minor issues with the paper, including the writing. There are some parts of the paper that are clear and well-written, while other parts need significant editing.
The following are more specific comments.
Abstract: Please have someone not familiar with the paper review the abstract.
Q1: Why do we need dynamic HDR lightmaps? What is their utility? Please start the abstract with some kind of motivation for the work. HDR is pretty common, but all acronyms should be defined with the first use.
Q2: Regarding the sentence: "Additional lighting states, external assets, and an Unreal Engine integration characterize temporal coverage and deployment costs." How do lighting states, assets, or an unreal engine characterize something? They can represent deployment costs, or be deployment costs, but they aren't agents that do things. What does temporal coverage mean in this context? Note that 'Unreal Engine' is not mentioned in the entire paper except in the abstract. Why is it there?
Q3: What is a 'matched-storage diagnostic' or 'weakest fitted-state quality'?
Q4: HEVC is not defined. But if HEVC does better, what is the purpose of this paper? The sentence says HEVC does better than Gaussian alternatives, but the paper is about using Gaussians. What is this supposed to tell the reader?
The abstract is the only thing most people will ever read about your paper. Explaining the problem you are solving, the key novel elements, and a short summary of how well it performs are the key things that need to come across. Including the ablation study results in the abstract is pretty much meaningless to anyone who hasn't already read the paper. I'd strongly recommend re-writing the abstract to provide more meaning and less detail. For example, the sentence
"A twelve-state, 512 ×1024 sequence occupies approximately 0.41 MiB versus 72 MiB in Float32, with 1.96 ms median full-frame reconstruction on an RTX 4090."
could be re-written as
"On a twelve-state, 512 ×1024 sequence, the proposed method achieves 175:1 memory compression with <2ms median full-frame reconstruction on an RTX 4090."
This avoids boring your reader with too many numbers and clarifies the impact of the method. Focus the abstract on meaning and impact.
The start of the introduction is much better.
One of the consequences of using fixed spatial Gaussians is that the kernel distribution and size must reflect the needs of the entire lighting sequence. That implies that in a lighting sequence with moving high frequency lighting, there would need to be a large number of almost uniformly small kernels. The paper uses a fixed number of kernels, which suggests the results may be at risk of missing high frequency information.
Lines 95:98 have one sentence with a colon and the next sentence with a semi-colon. While possibly grammatically correct, these patterns of writing create an uneven reading experience that does not help with understanding. The semi-colon in this example is gratuitous and could be replaced with 'so'.
Please use commas or conjunctions to connect phrases or else create separate sentences that flow together rather than asking the reader to continuously move down and up hierarchies of concepts or mentally insert connectors that should be there. There are a number of other examples, such as line 105, the caption of figure 1, the caption of table 1 (twice), the capture of figure 2 (twice), line 140, and line 143, and many others, that read like listening to someone who is always going off on tangents for half a sentence. Most of the text is much, much better than that and much easier to parse. If you want a person to read and understand the paper, please avoid repeated use of colons and semi-colons when they are not necessary.
Line 106-107: 'This inverse scaling does not make lossy reconstruction lossless.' reads like a throwaway sentence with no explanation. I assume this is because the representation uses a fixed number of bits per position, so there is always quantization in the reconstruction.
line 184: why log-domain reconstruction? It makes sense to me, but what is the reasoning behind it (or is there a citation?).
Lines 197-200: It would be helpful to scan through the text and avoid changing from active to passive voice in alternating sentences. Lines 197-200 start in active voice, smashes two different thoughts together with a semi-colon, moves from "We report" before the semi-colon, to "match-cap study instead counts" after the semi-colon, to "objective is assumed." in the final sentence. Active voice is a bit easier to read, but please choose one or the other.
Line 215: why use MiB, and not just the typical MB (megabyte)? It makes me think you're using some other unit.
Line 250: what does the following mean? "; we do not combine that rounded size with an inconsistent 0.53 bpp claim."
Line 251: 'denoted by asterisks." where? I assume this is in table 2, which is not referenced until several paragraphs later.
Line 253: 'three locally available scenes' what does 'locally' mean in this context?
Lines 252-257 :It's unclear what the paragraph 252-257 is describing because there is no reference to a table, a figure, or particular experiments. This paragraph seems to be important because it is describing the source of the 40-scene test set (author-adaptations, how, what adaptations?), that only five of the 40 scenes are providing BC6H (an acronym never identified in the text), and that some of the experiments use only three scenes. Some of these are clarified later in the paper, but reading the paper linearly results in confusion.
Line 297: 'The fixed 6:00 row illustrates' assume this is in Figure 2?
Results section:
A scene 17 result of 30.03 dB is reported in the appendix (assume for Lightmap-GS). It's referenced on line 298 of the paper without context. 'These fits do not explain' is meaningless. What fits? What is a fit? Why is one number being reported in the appendix and referenced in the paper, but a different number is used in table 2?
Table 5 caption: Please remove the semi-colon and insert 'with' or 'using'.
Lines 352-354: remove the semi-colons. Commas are appropriate and cleaner. More importantly, the connections between the ablation study table and the three clauses are not clear. For example, 'canoncial colors make the residual task easier' does not really match up with the meaning of 'Direct temporal color instead of residual'. The text in line 352 says using 'canonical colors' make things easier, but the table indicates that using residuals instead of temporal colors makes things easier. The other two are ok, if you can connect TriPlane to 'factorized features' and connect 'uniformly filling valid charts' to 'Uniform valid-region'. Why not use the same language in both places?
What is the overall message of the latency timing studies? The Lightmap-GS system has a low latency for small numbers of queries, but much higher for large numbers. Which tends to occur in practice? Is the latency for 65k queries fast enough?
Table 8 shows decreasing performance with longer temporal sequences. The paragraph in lines 416-424 duplicates the contents of table 8 but doesn't make the purpose of the text or the meaning of the results clear. Given earlier commentary, using 24 or 48 states includes 'lights off' time. Does that mean having to represent dark scenes is more challenging for the residual color network to learn, or is it the strong contrast between light and dark which may skew the spatial allocation of the Gaussians? Removing the existing paragraph (which just duplicates information from the table) and replacing it with an analysis or more enlightening summary would help. Understanding why the performance is decreasing is important.
Spatial Distribution of the Gaussians:
The primary learning representation in this paper is the distribution and training of spatial Gaussians and the globally learned color features. The spacial allocation is done in stages, with 50% generated at the beginning and 12.5% every 500 steps. The system uses a fixed number of kernels (10k).
As this is the critical innovation, the details on spatial placement need to be clear.
The initial set uses 'gradient-weighted placement with random exploration'.
- Gradients of what?
- Random exploration makes it sound like a process. Are they placed using a uniform random distribution, a Poisson distribution, jitter sampling, or using some other sampling process?
- 'Subsequent additions use reconstruction error to focus capacity'. Does it use reconstruction error as a deterministic placement metric (e.g. local peaks) or is the reconstruction error used as a pdf for sampling new locations?
Section 3.4 should include these types of details given the centrality of the Gaussian placements. There are no ablation studies on placement methods.
Given the placement loss function is L1, the losses are a simple average over all states. If one state has a substantially different spatial distribution, the L-1 loss penalizes the average, not the max or the squared errors. Therefore, a large error E in one state is equal to losses of size E/N across N states, which discourages spatial adaptation to cover outliers in contrast to an L-2 or L-inf loss metric. There is no ablation study of the loss error metric.
As noted by the authors, the generality of the proposed method is also limited by the size of the test set. Many of the results use only five, or even just one scene. Scene 17 is chosen by the authors for evaluating the results across states, but there is no discussion of whether Scene 17 is unique, typical, easy, or hard. Does it have properties that make it challenging?
Looking at Figure 2, Lightmap-GS struggles with fine details, both positive (small lit areas that are missing) and negative (small areas of darkness withinin lighter patches). These suggest the Gaussians are either not being placed well or that there are not enough of them to properly sample the higher frequency elements of the scene. This ties in with the concerns about how the Gaussians are being placed and the loss function being used to learn their positions and variances.