acceptodds
Under review as a conference paper at ICLR 2027

ECLAR: Event-Conditioned Visual Token Repair for Low-Light Vision-Language Models

Abstract

Low-light vision-language understanding depends on recovering the visual representations that support correct answers. We introduce ECLAR (Event-Conditioned Latent Alignment and Recovery), which uses synchronized events to repair RGB patch states inside a frozen vision encoder. Four residual adapters align representations with paired clean targets, retaining pretrained readout modules and the visual token count. Inference uses only low-light RGB and events. SDE-VQA-Bench pairs 8,357 reviewed questions with 1,838 frames from 91 sequences to evaluate representation and semantic recovery. With 4.89M parameters trained on SEE-600K, ECLAR leads four final-token recovery metrics against the evaluated restoration cascades, reaching 0.7817 cosine similarity to clean tokens. It improves frozen Qwen3-VL-2B by 11.91 points in six-family Macro and exceeds the strongest cascade by 4.19 points, recovering 57.2% of the clean-to-degraded gap. Matched events add 2.58 points over a capacity-matched RGB-only control; InternVL2.5-2B gains 4.41 points. These results connect clean-token alignment with recovery of both visual representations and downstream semantic performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Low-light Vision, Event-based Vision, Latent Alignment, Visual Token Repair, Parameter-Efficient Fine-Tuning, Visual Question Answering

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_1JCKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ECLAR repairs low-light RGB tokens inside a frozen vision encoder using event-conditioned residual adapters and clean-reference supervision while preserving the native visual interface. The authors introduce SDE-VQA-Bench with 8,357 questions. ECLAR improves Qwen3-VL-2B by 11.91 Macro points, with additional support from cross-backbone experiments and representation metrics.

Critical strengths
  1. The method trains only 4.89M parameters, preserves the native interface, and improves both Qwen3 and InternVL, demonstrating practical value.
  2. The authors evaluate both question-answering performance and visual representation recovery, and use an equal-parameter control to support the additional benefit of real event information.
Critical weaknesses
  1. Missing controlled ablations of single-site versus multi-site repair and supervision objectives, together with differing Event-MLLM training settings, make the core design’s contribution difficult to isolate
Questions for authors
  1. Under matched training conditions, how much does four-site repair improve over single-site repair and final-token fusion?
1 comment

Thank you for your careful review and constructive comments. We appreciate your emphasis on distinguishing the contribution of the repair architecture from that of the training settings. Your feedback has helped us strengthen the experimental analysis. Below, we provide additional comparisons and clarify the purpose and scope of the two Event-MLLM experiments.

1. Four-site repair versus single-site repair and final-token fusion

Following your suggestion, we added a direct architectural comparison with matched parameter budgets, backbone, training data, and optimization settings. The first four variants below share final-token supervision and auxiliary objectives (F+A), with intermediate representation alignment omitted. We additionally report the complete ECLAR result from the original manuscript, which includes intermediate alignment (M).

ArchitectureRepair/fusion locationMacro (%) ↑
Single-site repairLayer 160.86
Single-site repairLayer 464.45
Four-site repair[1, 2, 3, 4]66.09
Final-token fusionAfter the visual merger55.40
ECLAR (full M+F+A; original manuscript)[1, 2, 3, 4]66.86

Under shared F+A supervision, four-site repair improves over single-site repair at layer 1 by 5.23 percentage points, single-site repair at layer 4 by 1.64 points, and final-token fusion by 10.68 points. Differences are calculated from unrounded scores. These controlled comparisons support the benefit of the tested multi-site in-encoder repair architecture. The full ECLAR result is included as a reference for the complete method.

2. Event-MLLM training settings and comparison protocols

Thank you for highlighting the importance of comparable training conditions. Our manuscript includes two complementary Event-MLLM experiments: an official-code reproduction and a separate retraining experiment that aligns the backbone and training data with ECLAR. We clarify their respective purposes below.

Official-code reproduction — Appendix H and Table 2(c). We reproduce both training stages using Qwen2.5-VL-3B, the released training data, and the released-code configuration. To assess reproduction fidelity, we also evaluate the reproduced model on the original tasks using the official inference and scoring routines.

As reported in Table 14, the reproduced multiple-choice accuracy is 32.90%, compared with the published 34.48%, and multiple-choice F1 is 79.78%, compared with 80.35%. Counting accuracy is 72.06%, compared with 71.08%, while counting MAE is 0.5834, compared with 0.6127. These results support comparable original-task performance. This experiment is reported separately because its backbone and training-data exposure differ from those of ECLAR.

Backbone- and data-aligned retraining — Appendix B.5 and Table 2(a). We additionally retrain the public Event-MLLM alignment-stage architecture using the same Qwen3-VL-2B backbone and SEE-600K training data as ECLAR, with both trained for three epochs. Both settings freeze the VLM backbone and exclude language-model instruction tuning.

This experiment provides a comparison using the same backbone and training data while retaining method-specific representation objectives, preprocessing, and optimization settings. These differences are documented in Appendix B.5. Accordingly, we present it as a method-level comparison, rather than attributing the entire performance gap to fusion location alone.

Together, the two experiments provide evidence of faithful reproduction and a comparison with aligned backbone and training data. We will make their distinction and the remaining training differences more prominent in the revised manuscript so that the scope of each comparison is clear.

Thank you again for helping us improve the clarity and rigor of the paper. We hope these results and explanations clarify the comparisons, and we would appreciate your feedback on any concerns that remain so that we can address them during the discussion.

Reviewer_HyCzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ECLAR, an event-conditioned visual token repair framework for improving vision-language model (VLM) understanding in low-light conditions. Rather than restoring degraded RGB images in pixel space, ECLAR injects synchronized event information into the intermediate representations of a frozen vision encoder via lightweight residual adapters. These adapters are trained to align degraded visual representations with those extracted from clean reference images, while keeping both the original visual token interface and the pretrained model parameters intact. The authors further introduce SDE-VQA-Bench, a benchmark of 8,357 questions spanning seven task categories, to evaluate semantic recovery under low-light conditions. Experiments on Qwen3-VL and InternVL show consistent improvements over image restoration baselines, and additional evaluations on event-based driving tasks further support the broader applicability of the approach.

Critical strengths

The paper addresses an important limitation of conventional low-light enhancement pipelines, which optimize for image reconstruction quality without directly accounting for downstream semantic understanding. Repairing intermediate visual representations offers a compelling alternative to pixel-space restoration. The approach is also parameter-efficient, requiring only 4.89M trainable parameters while keeping the pretrained VLM frozen.

Critical weaknesses
  1. The proposed framework largely integrates established techniques, including event feature extraction, residual adapters, and clean-reference representation distillation. While this combination is reasonable and practically motivated, the methodological novelty beyond these existing components appears somewhat limited. In addition, the paper lacks sufficiently detailed ablations of the individual training objectives and architectural choices. For instance, separately evaluating the effects of intermediate representation alignment, final-token supervision, and the auxiliary losses would help clarify the contribution of each component and identify which elements are essential to the reported performance gains.

  2. As shown in Table 4, the RGB-only control already improves Macro accuracy from 54.95 to 64.28, while incorporating matched events further increases it to 66.86. This suggests that a substantial portion of the overall improvement may stem from representation adaptation rather than from event-specific information. Although the additional gain from matched events is still meaningful, the current experiments do not fully establish how effectively ECLAR exploits complementary information from the event modality. Additional evaluations with mismatched, noisy, or temporally misaligned events would provide stronger evidence for the utility of event information and help characterize the robustness of ECLAR under more realistic conditions.

  3. SDE-VQA-Bench is constructed primarily using model-generated annotations and automated auditing, with human adjudication applied only to cases involving model-label disagreements. Such a pipeline may introduce systematic annotation biases that are not fully captured by the current validation procedure. Moreover, the benchmark is derived from a single source dataset, which limits the evidence for generalization across different event sensors, capture conditions, and low-light environments. The paper also selects the main adapter placement based on performance on SDE-VQA-Bench, which may introduce evaluation-set selection bias. An independently annotated evaluation set, together with a separate validation protocol for architectural selection, would strengthen the reliability of the reported conclusions.

Questions for authors
  1. Could the authors provide additional ablations separating the effects of intermediate representation alignment, final-token supervision, and auxiliary training objectives? Which components contribute most significantly to the performance improvements?

  2. What proportion of SDE-VQA-Bench annotations were manually verified? Could the authors provide an estimate of annotation accuracy based on a randomly sampled, independently human-annotated subset, including cases where the annotation models initially agreed?

4 comments
AuthorsOpenReview ↗

Response 1/4 — W1–W2

Thank you for your thoughtful assessment. We appreciate your suggestions for clarifying the methodological contribution and examining how ECLAR uses event information. Below, we provide architectural ablations and controlled event interventions. Detailed training-objective ablations are reported in our response to Q1.

W1. Methodological contribution and architectural ablations

ECLAR's contribution lies in event-conditioned repair along a frozen visual encoder: synchronized events guide residual updates at multiple intermediate locations, while the pretrained readout modules and visual token count remain unchanged. This formulation builds on established event encoders, residual adapters, and representation distillation. We will sharpen the contribution statements around the repair formulation and its empirical benefits.

To examine the contribution of the repair configuration, we conducted architectural ablations using the same final-token supervision and auxiliary objectives, with approximately matched trainable-parameter budgets:

ConfigurationRepair/fusion locationSDE Macro (%) ↑
Single-site repairLayer 160.86
Single-site repairLayer 464.45
Four-site repairLayers [1, 2, 3, 4]66.09
Final-token fusionAfter the visual merger55.40

Under this shared objective, four-site repair improves Macro by 5.23 percentage points over layer-1 repair, 1.64 points over layer-4 repair, and 10.68 points over post-merger fusion. The four-site configuration in this table excludes intermediate alignment, so these gains establish the advantage of the tested repair configuration under shared supervision.

Our response to Q1 further separates the contributions of intermediate alignment, final-token supervision, and auxiliary objectives with the architecture fixed.

W2. Benefits of correctly matched event information

The original capacity-matched comparison shows that matched events improve Macro from 64.28% for the RGB-only control to 66.86%, a gain of 2.58 percentage points. This measures the additional benefit of event information beyond the gains already obtained through RGB representation adaptation.

Following your suggestion, we further tested whether ECLAR uses events corresponding to the current RGB observation. We fixed the ECLAR checkpoint, RGB inputs, questions, and decoding settings, and changed only the event inputs.

All non-temporal interventions use 8,357 questions, with a matched-event baseline of 66.84% Macro. Both temporal shifts use the same 8,141 valid questions, with their matched-event baseline of 66.81%.

Event interventionMacro (%)Change (pp)
Events from another sequence38.42−28.42
Spatial mirroring of events40.27−26.56
Polarity inversion65.05−1.78
Add 25% spurious events65.80−1.04
Add 50% spurious events64.82−2.02
Add 100% spurious events63.53−3.31
Remove 25% of events66.56−0.28
Remove 50% of events66.33−0.50
Remove 75% of events65.16−1.67
Temporal shift: −3 frames62.26−4.56
Temporal shift: +3 frames64.80−2.01

The additive-noise percentage denotes newly injected spurious events divided by the original event count; all original events are retained. Values are rounded for display.

These controls provide complementary evidence:

  • Scene and spatial correspondence: Replacing matched events with events from another sequence reduces Macro by 28.42 points; spatially mirroring events reduces it by 26.56 points. With the RGB input and model fixed, these changes demonstrate that event correspondence materially affects the model's predictions. Together with the RGB-only comparison, they support the contribution of correctly matched event information.
  • Temporal correspondence: Shifting events by −3 or +3 frames reduces Macro by 4.56 and 2.01 points, respectively, on the common valid subset. Thus, temporally matched events yield better performance than the tested shifted inputs.

The RGB-only control establishes the positive gain from matched events, while the fixed-model interventions show the importance of their correspondence with the RGB observation. Together, these results strengthen the evidence that ECLAR benefits from scene-specific event information. We will incorporate these comparisons and their interpretation into the revised manuscript.

AuthorsOpenReview ↗

Response 2/4 — W3

W3. Annotation reliability, evaluation beyond SDE, and placement selection

Annotation reliability. We agree that adjudicating model disagreements alone cannot detect errors shared by both annotation models. We therefore conducted an additional human audit of 400 questions selected through proportionate random sampling from model-agreement cases. Together with the 795 manually adjudicated questions already included in the released benchmark, human review now covers 1,195 of 8,357 questions (14.30%). Our response to Q2 provides the task-level audit and shows that ECLAR retains its Macro advantage when the audited subset is scored against human reference answers.

Evaluation beyond SDE. We additionally evaluate the repair architecture on EventDrive (Lu et al., 2026), which integrates DSEC (Gehrig et al., 2021), M3ED (Chaney et al., 2023), and PKU-DAVIS-SOD (Li et al., 2023). Our Table 7 includes Regular and Hard results, with perception evaluations across all three sources. This provides additional evidence for the applicability of the repair architecture across different acquisition setups and driving tasks. The EventDrive experiments use separate task-specific supervised training and complement the SEE-to-SDE representation-recovery study.

Placement selection and effectiveness across repair locations. As reported in Section 5.2 and Table 6, all six evaluated layouts improve SDE Macro over Degraded RGB by 10.34–11.91 percentage points, with an average Macro of 66.10%. These gains span both early and readout-adjacent placements, supporting the effectiveness of ECLAR across the tested insertion locations.

Section 5.2 also distinguishes the preferred layouts under different aggregation metrics: [1, 2, 3, 4] leads Macro, while [4, 10, 16, 22] leads Micro. Using SDE Micro as the selection criterion therefore selects [4, 10, 16, 22]. The following table presents the four representation metrics evaluated on the held-out SEE validation split, alongside SDE Macro and Micro.

Repair locationsSEE Cosine ↑SEE Relative L2 ↓SEE Linear CKA ↑SEE Relation error ↓SDE Macro (%) ↑SDE Micro (%) ↑
[5, 11, 17, 23]0.75800.73090.38360.128665.2974.25
[4, 10, 16, 22]0.75750.73060.38750.127465.7175.12
[4, 10, 16, 23]0.75950.72590.38350.128965.6074.44
[1, 2, 3, 5]0.74540.75310.37490.136866.4275.08
[1, 2, 3, 4]0.74850.74550.38230.133666.8675.10
[0, 1, 2, 3]0.74650.74960.37630.135866.7275.01

The Micro-leading layout also achieves the highest SEE CKA and lowest SEE token-relation error among the six candidates. This configuration therefore combines strong preservation of clean-reference representation structure with strong downstream semantic performance, complementing the joint representation and semantic recovery discussed in Section 5.1. It achieves 65.71% SDE Macro, retaining gains of 10.76 points over Degraded RGB and 3.04 points over the strongest cascade, DA-CLIP official.

The original manuscript also addresses the uncertainty associated with placement selection: Appendix C explicitly notes that the reported confidence interval does not include architecture-selection uncertainty. Our central conclusion is supported by the consistent improvements across all six tested layouts: ECLAR’s effectiveness is not confined to the particular insertion layout used in the main table. We will make the selection criteria and this scope of the placement comparison more explicit in the revised manuscript.

Thank you again for these suggestions. They have helped us clarify the contribution, strengthen the evidence for event conditioning, and better delineate the scope of the evaluation.

AuthorsOpenReview ↗

Response 3/4 — Q1

Thank you for these specific questions. Below, we provide the additional training-objective ablations and human-verification results.

Q1. Could the authors separate the effects of intermediate alignment, final-token supervision, and auxiliary objectives? Which components contribute most?

We denote intermediate representation alignment by M, final-token supervision by F, and the auxiliary training objectives collectively by A. With the repair architecture fixed at four sites [1, 2, 3, 4], we obtain:

Training objectivesLow-light SDE Macro (%) ↑Normal-light RGB intervention Macro (%) ↑
M only64.1073.71
F + A, without M66.0975.03
M + F, without A67.0974.52
Full M + F + A66.8675.44

These comparisons distinguish the following conditional contributions:

  • Final-token supervision: Adding F to M, with A absent, improves low-light Macro by 3.00 percentage points, the largest positive low-light gain among the tested objective contrasts.
  • Intermediate alignment: Adding M to F + A provides a further 0.77-point improvement in low-light Macro.
  • Auxiliary objectives: Adding A to M + F decreases low-light Macro by 0.24 points, while improving normal-light RGB intervention Macro by 0.92 points.

Differences are calculated from unrounded scores.

Thus, final-token supervision provides the largest observed marginal improvement in these comparisons, while intermediate alignment contributes an additional benefit. The auxiliary objectives exhibit a trade-off: M + F achieves the highest low-light Macro, whereas the full model achieves the best normal-light intervention performance among the tested trained variants.

Design motivation of A and normal-light preservation. An explicit design goal of ECLAR is to improve low-light understanding while avoiding unnecessary corrections and substantial degradation on normal-light inputs. This goal is reflected in the clean-input identity objective, already included in Section 3.3, Eq. (6) and defined in Appendix B.3, Eq. (9). It encourages repaired representations of clean RGB with corresponding clean-condition events to remain close to the frozen clean-reference representations.

Normal-light RGB intervention on SDE. We retain the original low-light events and replace only the RGB input with its normal-light counterpart. Results on all 8,357 questions are:

ModelMacro (%) ↑Difference from RGB Base (pp)
Native RGB Base75.7579—
M + F + A75.4437−0.3142
M + F, without A74.5248−1.2331

Adding A improves Macro by 0.9189 percentage points, reducing the deficit to the native RGB Base from 1.2331 to 0.3142 points.

Normal-light diagnostics on SEE. We additionally evaluate 499 unique images, using normal-light RGB and its corresponding normal-light events:

Diagnostic metricWith A (M + F + A)Without A (M + F)
Mean gate activation, averaged over four layers0.04390.7546
Relative repair residual, averaged over four layers1.57%26.32%
Final-token relative L2 shift ↓0.09890.6046

Together, the answer-level results and representation diagnostics support A's role in suppressing unnecessary repair and preserving normal-light representations, providing an empirical rationale for retaining A despite the small low-light trade-off. This comparison assesses A as a whole and does not isolate the gate regularization term.

We will revise the discussion to reflect these distinct roles rather than suggesting that every objective uniformly improves low-light accuracy. The architectural comparisons using shared F + A supervision are provided in our response to W1.

AuthorsOpenReview ↗

Response 4/4 — Q2

Q2. What proportion of annotations were manually verified, and what is their accuracy on a randomly sampled human-annotated subset, including model-agreement cases?

Human verification coverage. Of the 8,357 questions in the released benchmark, 795 were manually adjudicated during construction because the two annotation models disagreed.

To assess errors that could remain when both models agreed, we additionally selected 400 questions through proportionate random sampling from model-agreement cases for human verification. This subset covers 80 sequences, 357 frames, and all seven task types. All 400 questions originally belonged to the automatically agreed-upon group.

The two groups are disjoint. Therefore, 1,195 questions, or 14.30% of the released benchmark, have received human review.

Agreement with human reference answers. In the newly audited subset, the original answers agree with the human references on 365 of 400 questions (91.25%), with 35 disagreements (8.75%):

TaskAudited questionsAgreement with human referenceDisagreementsAgreement rate
Attribute5248492.31%
Counting1812666.67%
Object Presence9084693.33%
Object Recognition7671593.42%
Scene Understanding8076495.00%
Text Presence7570593.33%
OCR Transcription94544.44%
Total4003653591.25%

The audit identifies remaining weaknesses in Counting and OCR. Their audited sample sizes—18 and 9 questions, respectively—also limit the precision of task-specific agreement estimates. We will report both the observed discrepancies and this limitation. The overall 91.25% is the observed agreement rate for the randomly sampled model-agreement cases.

Effect on model comparisons. We further evaluated the same four methods on this 400-question subset using both the original labels and the human reference answers:

MethodMacro with original labels (%) ↑Macro with human references (%) ↑Per-question accuracy with human references (%) ↑
Degraded RGB60.5656.0664.75
RGB-only control72.6867.0175.50
DA-CLIP official73.5568.4074.00
ECLAR, fully retrained77.5671.7877.00

Using human references lowers the absolute Macro scores of all four methods, confirming that annotation quality affects absolute benchmark performance. Nevertheless, the Macro ranking remains unchanged. ECLAR retains improvements of 15.72 percentage points over Degraded RGB, 4.77 points over the RGB-only control, and 3.38 points over DA-CLIP official.

These results support ECLAR’s comparative advantage on the human-audited subset while identifying annotation weaknesses that require correction. We will include the sampling protocol, task-level agreement rates, and results under human references in the revised manuscript.

Thank you again for helping us strengthen the experimental analysis and benchmark validation. We would welcome your feedback on any concerns that remain.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to W1 / Q1

Thank you for the detailed review. We have added paired uncertainty estimates and inference-time event interventions to distinguish representation adaptation from the additional contribution of matched events.

W1 / Q1: Paired uncertainty and the RGB-only control

For the original main-table checkpoints, paired sequence-bootstrap estimates give:

ComparisonMacro gain (pp)95% CI
ECLAR vs RGB-only control+2.5803[+0.7680, +4.4868]
ECLAR vs DA-CLIP official+4.1871[+1.0537, +7.1734]

Both intervals are above zero for the evaluated checkpoints. These comparisons quantify the additional matched-event gain and the advantage over the strongest cascade. The RGB-only control also establishes that clean-reference representation adaptation contributes substantially to the overall improvement.

We verified the control implementation: event voxels are replaced with ones before event encoding and statistic computation. Consequently, with five temporal bins, s_i is constant at [ln(6), ln(6), 0], and presence is always one. Sample-specific event information is therefore removed while retaining the repair architecture. In the real-event branch, presence indicates whether the image’s event stream is nonempty and is broadcast across tokens; the learned repair gates are patch-dependent.

W1, W7: Matched event correspondence and robustness

We additionally fix the complete retrained ECLAR checkpoint, RGB inputs, questions, and decoding, changing only the event input:

Event interventionQuestionsMatched MacroIntervention MacroChange (pp)
Cross-sequence mismatch8,35766.8438.42−28.42
Spatial mirroring8,35766.8440.27−26.56
Polarity inversion8,35766.8465.05−1.78
Add 25% spurious events8,35766.8465.80−1.04
Add 50% spurious events8,35766.8464.82−2.02
Add 100% spurious events8,35766.8463.53−3.31
Drop 25% of events8,35766.8466.56−0.28
Drop 50% of events8,35766.8466.33−0.50
Drop 75% of events8,35766.8465.16−1.67
Temporal shift: −3 frames8,14166.8162.26−4.56
Temporal shift: +3 frames8,14166.8164.80−2.01

Scores are percentages; differences use unrounded values. Both temporal shifts use the same 8,141 valid questions. Spurious-event percentages denote added events divided by original events; all original events are retained. The retrained intervention checkpoint scores 66.8359, while the original main-table checkpoint scores 66.8558.

Correctly matched events support the repaired representation: changing their scene or spatial correspondence substantially reduces performance despite unchanged RGB and model weights. Together with the controlled RGB-only comparison, this supports the use of sample-specific event evidence. The intervention drops measure sensitivity to corrupted inputs, while the RGB-only contrast measures the incremental benefit of matched events. Performance changes under noise, dropout, and temporal shifts further characterize the operating conditions of the model.

For clean-input preservation, our objective ablations below report normal-light evaluation and repair-magnitude diagnostics. Regarding backbone transfer, Appendix F already specifies that InternVL uses [1,2,3,4] without an InternVL-specific placement search. The EventDrive experiments provide separate evidence under task-specific training; they do not constitute zero-shot transfer across capture setups.

We will include the full intervention results and protocols in the appendix and update Appendix C with the two additional paired contrasts.

W2 / Q2: Controlled single-site, four-site, and final-token comparisons

We have added a direct architectural comparison with matched parameter budgets, backbone, training data, and optimization settings. The first four variants below use the same final-token supervision and auxiliary objectives (F+A), with intermediate alignment omitted. We also include the complete ECLAR model, which additionally uses intermediate representation alignment (M).

ArchitectureRepair/fusion locationMacro (%)
Single-site repairLayer 160.86
Single-site repairLayer 464.45
Four-site repair[1,2,3,4]66.09
Final-token fusionAfter the visual merger55.40
ECLAR (full M+F+A)[1,2,3,4]66.86

Under the shared F+A supervision, four-site repair improves over layer 1 by 5.23 points, layer 4 by 1.64 points, and final-token fusion by 10.68 points. These comparisons isolate the tested repair architectures under a shared supervision setting. The additional contribution of intermediate alignment in the full ECLAR model is examined in the objective ablation below.

The two Event-MLLM experiments serve different purposes. Appendix B.5 / Table 2(a) retrains its public alignment-stage architecture with the same Qwen3-VL-2B backbone and SEE-600K data as ECLAR, for three epochs with the VLM frozen. Appendix H / Table 2(c) reproduces the official two-stage setup using Qwen2.5-VL-3B and the released training data, including SDE-source exposure. The latter is a reproduction experiment. The new controlled fusion comparison above supplies the direct evidence for repair location; the cascade comparison evaluates complete systems with their respective training objectives.

W8, W7: Training objectives and normal-light preservation

We fix the four repair sites and all other training settings. M denotes intermediate representation alignment, F final-token supervision, and A the auxiliary objectives.

Training objectivesLow-light Macro (%)Clean-RGB intervention Macro (%)
Native RGB baseline—75.7579
M only64.096873.7140
F+A, without M66.085175.0297
M+F, without A67.094274.5248
Full M+F+A66.855875.4437

Adding F to M improves low-light Macro by 3.00 points. Adding M to F+A provides 0.77 points. Adding A to M+F changes low-light Macro by −0.24 points, while improving normal-light Macro by 0.92 points. Thus, F produces the largest gain among these tested conditional contrasts, M adds further recovery, and A supports normal-light preservation.

Preserving useful normal-light inputs is part of the design motivation for the clean-input identity objective (Section 3.3 and Appendix B.3). The SDE intervention replaces only the RGB image with its normal-light counterpart and retains the original low-light events across all 8,357 questions. The full model is 0.3142 points below the native RGB baseline, compared with 1.2331 points without A.

A separate diagnostic on 499 independent SEE images with their corresponding normal-light events gives:

DiagnosticWith AWithout A
Mean gate activation, averaged over four sites0.04390.7546
Relative repair residual, averaged over four sites1.57%26.32%
Final-token relative L2 deviation0.09890.6046

These results support A’s collective role in limiting unnecessary corrections on normal-light inputs. They do not assign that effect to an individual auxiliary term.

W9: Interpretation of representation recovery

Thank you for this helpful comment. We agree that the relationship between representation similarity and downstream performance should be described as an association. We will adopt your suggestion in the revised main text, clarifying that individual representation metrics need not monotonically predict VQA performance and that the token-similarity and VQA recovery fractions measure different quantities.

W3 / Q3: Placement selection and consistency across layouts

The central placement result is that recovery occurs across all six tested layouts. Section 5.2 explicitly states that the original main layout [1,2,3,4] was chosen for its leading SDE Macro, whereas [4,10,16,22] leads Micro. Appendix C also states that the reported interval excludes architecture-selection uncertainty.

The table below combines the original SDE results with additional representation diagnostics on a held-out SEE validation split. Best values in each column are bolded.

Repair locationsSEE Cosine ↑SEE Relative L2 ↓SEE CKA ↑SEE Relation error ↓SDE Macro (%) ↑SDE Micro (%) ↑
[5,11,17,23]0.75800.73090.38360.128665.2974.25
[4,10,16,22]0.75750.73060.38750.127465.7175.12
[4,10,16,23]0.75950.72590.38350.128965.6074.44
[1,2,3,5]0.74540.75310.37490.136866.4275.08
[1,2,3,4]0.74850.74550.38230.133666.8675.10
[0,1,2,3]0.74650.74960.37630.135866.7275.01

Layout-aggregated results. Across the six layouts, mean Macro is 66.10% and mean Micro is 74.83%. Macro ranges from 65.29% to 66.86%. Every layout improves over Degraded RGB by 10.34–11.91 Macro points. The mean Macro improvement is approximately 11.15 points over Degraded RGB and 3.43 points over DA-CLIP official. These are descriptive aggregates of the evaluated layouts.

The Micro-leading layout [4,10,16,22] achieves 65.71 Macro / 75.12 Micro, retaining approximately 10.76 Macro points over Degraded RGB and 3.04 points over DA-CLIP official. It also achieves the highest SEE CKA and lowest SEE relation error, combining strong clean-reference structural alignment with strong downstream performance.

These results support the effectiveness of repair across early and readout-adjacent locations. We interpret the smaller differences between individual layouts cautiously. The held-out SEE measurements provide supplementary representation diagnostics; the main layout remains the original SDE-selected configuration. The consistency across layouts supports the method’s broader placement applicability without resolving the selection uncertainty of that particular main configuration.

We will add the full diagnostic table and layout aggregates to the appendix and make this distinction explicit in the placement discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.