- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
HazeMT-DCR addresses multi-task prediction in hazy scenes without explicit image restoration. It combines degradation-aware feature modulation, instruction-conditioned shared experts, and task-specific reinforcement-learning routing. Expert-removal and task-demand-replacement counterfactual losses further guide training. Experiments on synthetically hazed PASCAL-Context report improved overall performance with 5.08M trainable parameters. Incremental adaptation experiments, including an additional detection task on real hazy RTTS images, evaluate reuse of frozen shared modules.
Critical strengths
- The framework directly addresses task-dependent feature requirements under degradation. Counterfactual expert removal and instruction replacement provide concrete training signals beyond the final prediction loss, and the reported ablations support their usefulness.
- The evaluation covers four dense prediction tasks, multiple task-expansion configurations, and real-haze object detection. The reported performance–parameter tradeoff is promising, and the real-data extension strengthens the practical relevance.
Critical weaknesses
- The incremental evaluation needs clarification. The paper states that all shared modules and existing task paths remain frozen, yet old-task results change after expansion (Tables 2 and 4). With identical inputs, instructions, model states, and deterministic inference, those predictions should remain unchanged. Changes in instruction composition or evaluation conditions could explain this, but must be documented to interpret the retention results.
- The benefits of the optimization components are not fully isolated. The actor receives both PPO and direct reparameterized gradients, while the counterfactual losses also update the expert network. Although a Softmax-routing comparison is provided, it remains unclear whether PPO adds value over the same stochastic router trained only through differentiable losses. Moreover, if gradients flow through counterfactual losses, regret could decrease by worsening counterfactual predictions rather than improving the factual prediction.
Questions for authors
- What causes old-task predictions to change when their entire computation paths are frozen? Are instructions, normalization statistics, and evaluation inputs identical before and after expansion?
- Does PPO improve over reparameterized Dirichlet routing trained with the same task and counterfactual objectives but without PPO? Please include training time and repeated-run variability.
- Are the counterfactual comparison losses detached when optimizing ? Can you show that reduced regret corresponds to better factual predictions rather than degraded counterfactual branches?
3 comments
- Does PPO improve over reparameterized Dirichlet routing trained with the same task and counterfactual objectives but without PPO? Please include training time and repeated-run variability.(Weakness 2 ,Question 2)
We agree that the original Softmax routing comparison changes both the routing parameterization and the optimization method, and is therefore insufficient to isolate the contribution of PPO. To address this, we have added a Dirichlet Routing w/o PPO control to the revised manuscript and report the mean performance, standard deviations, and average training time across three runs in Table 7. The results show that, while retaining Dirichlet reparameterized sampling and the direct gradients from the task loss and counterfactual loss, adding PPO updates further improves the mean performance of all four tasks, with a 7.49% increase in the time cost of the full training procedure. Details are provided below:
The newly added control retains the same Dirichlet routing formulation as the full model. Both configurations use reparameterized sampling and retain the gradients from the task loss and counterfactual loss that propagate to the actor through the expert weights . The control without PPO still includes stochastic routing exploration and can directly learn the routing policy through differentiable losses; it also retains the original counterfactual supervision. To control for the effects of initialization and training randomness, the two configurations use matched warm-up checkpoints and random seeds. Apart from removing PPO policy updates, the routing parameterization, expert and prediction modules, task and counterfactual supervision, and established training settings remain the same. The newly added control disables PPO policy updates while retaining the use of the aforementioned differentiable objectives to train the actor and prediction network. Both configurations use the same number of training epochs: a total of 400 epochs, consisting of 200 epochs of expert warm-up and 200 epochs of joint optimization, with a batch size of 16. The experiments are conducted on a single NVIDIA GeForce RTX 4090 (24 GB). Therefore, the performance comparison is based on the same training epoch budget, while differences in actual elapsed time are reported separately.
As shown in Table 7, the full model improves the mean performance on every task compared with reparameterized Dirichlet routing without PPO.
Because the newly added control already uses the same Dirichlet reparameterized sampling and receives direct supervision from the task and counterfactual losses, these results support the additional empirical benefits of PPO beyond direct differentiable optimization. This conclusion no longer depends on the original Softmax routing comparison.
Regarding variability across repeated runs, we also note that the full model has higher standard deviations than the control without PPO on all four metrics. Thus, this experiment supports improvements in mean task performance, but it does not justify a claim that PPO reduces variance across runs.
We have clarified reparameterized sampling and PPO gradient handling in Sections 3.4–3.5. Appendix A.2 explains that the actor receives both direct differentiable gradients and PPO gradients; Appendix A.3 provides the training settings; and the "Isolating the Contribution of PPO" paragraph in Appendix A.5 and Table 7 present the newly added control, variability across three runs, and training time. We thank the reviewer for this suggestion, which has enabled more direct and transparent verification of PPO's incremental contribution and training cost.
- Are the counterfactual comparison losses detached when optimizing ? Can you show that reduced regret corresponds to better factual predictions rather than degraded counterfactual branches?(Weakness 2 ,Question 3)
To first answer the question about gradient settings directly: when optimizing , the original method does not detach either the expert removal loss or the task demand replacement loss . The Counterfactual Regret Loss remains differentiable, allowing the expert modules to be updated and gradients to propagate to the actor network through the reparameterized expert weights.
We understand the reviewer's concern: the identification and correction mechanism described above does not imply that every decrease in the hinge objective necessarily results from improvements in factual predictions. When gradients are retained for the counterfactual losses, it is indeed necessary to examine whether regret is reduced by increasing the counterfactual losses. We have therefore added the following analyses that directly examine the optimization behavior.
In the Counterfactual Optimization Analysis in Appendix A.5, we use the same batch of 64 validation images to evaluate the checkpoints at epochs 201 and 400, reporting both types of counterfactual regret, the three types of branch losses, and task performance.
The results in Table 10 show that both types of regret decrease for all four tasks, while task performance based on factual predictions improves for all four tasks. In addition, the factual loss and both types of counterfactual losses decrease for all tasks, and within each task, the absolute decrease in the factual loss is greater than the absolute decrease in either type of counterfactual loss. In particular, the loss of the task demand replacement branch, which the reviewer specifically highlighted, also decreases for all four tasks, with no overall increase.
These results show that, for the reported fixed validation subset and the two checkpoints evaluated, reductions in regret are accompanied by lower factual prediction losses and improved task performance, while both types of counterfactual predictions also improve. Thus, the observed results do not support the explanation that "regret is reduced solely by degrading counterfactual predictions overall," and provide direct empirical support for the effectiveness of the expert learning and combination correction mechanism.
Following the reviewer's suggestion, Table 6 in the revised manuscript compares the original full model with the TICR w/ Detached CF Losses variant. In the latter, gradients are stopped for both types of counterfactual prediction losses when constructing , while gradients for the factual prediction loss are retained.
The results in Table 6 show that the original model performs better on all four tasks. This indicates that, under the experimental settings in this paper, retaining counterfactual gradients helps achieve better final task prediction performance.
We need to clarify that what we refer to as a "harmful contribution" applies only to the current sample, task, and weight configuration: if the loss decreases after an expert is removed and the remaining weights are renormalized, this indicates that the current combination has room for improvement; it does not mean that the expert is harmful to all samples and tasks. During training, the task loss and the differentiable counterfactual loss jointly optimize the experts and the actor, while PPO further optimizes the routing policy. Thus, this mechanism adjusts both expert responses and how they are combined to improve factual predictions, rather than merely reducing the probability of a particular combination, and it does not guarantee that every update will reduce the weight of a particular expert.
Expert removal provides an explicit criterion for identifying harmful contributions within the current combination, while the regret penalty and PPO feedback provide an optimization mechanism for correcting expert combinations. The newly added Tables 10 and 6 provide empirical support through an analysis of optimization behavior and a comparison of gradient settings, respectively. We do not extrapolate these results into a guarantee of monotonic improvement for every update or every sample. However, they support the view that TICR uses counterfactual diagnosis to improve expert combinations and factual predictions, rather than relying on overall degradation of the counterfactual branches.
We thank the reviewer for the careful assessment and for highlighting the need to clarify incremental evaluation and better isolate the effects of PPO and counterfactual optimization. Below, we address, in order, old-task prediction consistency, the contribution of PPO beyond differentiable Dirichlet routing, and whether reduced counterfactual regret is accompanied by improved factual predictions.
We need to clarify that, for our method, the old-task evaluation protocols before and after expansion are not exactly the same. The two settings use the same evaluation state but different task instructions and different checkpoints, one before expansion and one after expansion; the parameters of the shared modules and existing task modules remain identical. Specifically: Old-task Only stage: We train the model using only old-task data and evaluate its predictive performance using the combined instruction for the old tasks. Incremental Adaptation stage: Starting from the trained old-task model, we freeze HASE, TCM, ICFE, and the routing networks and output heads of all existing tasks, introduce and train only the corresponding task-specific modules for the newly added tasks, and evaluate its predictive performance using the expanded four-task combined instruction. Therefore, the old-task evaluation protocols before and after expansion are not exactly the same.
Regarding normalization statistics, Our model uses no BatchNorm: LayerNorm is applied in the Swin backbone and expert control vectors, GroupNorm in the degradation estimator, FPN, decoder, and prediction heads, and no normalization in the PPO router. Thus, no running statistics require synchronization. Validation uses fixed 448×448 inputs with ImageNet normalization and no random augmentation. All evaluations use
eval(),torch.no_grad(), and deterministic Dirichlet-mean routing viaact_deterministic(), without sampling.We emphasize that the purpose of using different task combination instructions before and after expansion is to measure end-to-end performance under actual task expansion conditions. Before expansion, the model only needs to perform the original tasks, so it is evaluated using the combined instruction for the old tasks. After expansion, however, the model should be capable of performing all four tasks simultaneously. We therefore evaluate the expanded model using a joint four-task instruction and examine how well it retains its performance on the old tasks while performing all tasks. Compared with consistently using the old-task instruction, this evaluation better reflects the model's performance under full multitask execution conditions after new tasks are added.
To distinguish prediction preservation under a fixed instruction from performance changes under actual joint four-task execution conditions, we have followed the reviewer's suggestion and added the Old-Instruction Evaluation setting to Table 4 in the revised manuscript. We have reevaluated our model using the following three settings:
The experimental results in Table 4 of the revised manuscript show that, across all ten task expansion settings, the old-task performance is exactly the same under Old-task Only and Old-Instruction Evaluation. Moreover, with the same input images and task instructions and under deterministic inference conditions, the maximum absolute difference in the raw prediction outputs is 0, further verifying the consistency of old-task predictions under fixed evaluation conditions.
In contrast, Incremental Adaptation uses the four-task combined instruction, so old-task performance may change. These control results show that, across the ten task expansion settings tested, both old-task performance and raw prediction outputs remain identical when the same old-task instruction is used before and after expansion. However, for the same expanded checkpoint, switching from the old-task instruction to the four-task combined instruction may change old-task performance. Therefore, the performance changes in the original table can be explained by the change in the task combination instruction.
We have further clarified the relevant evaluation protocols in Section 4.3 and Appendix A.4 of the revised manuscript and added the fixed-instruction control results to Table 4 to resolve any ambiguity in the original text.