- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AffectRAG, a continuous affective image editing framework that aims to achieve target Valence-Arousal (V-A) coordinates. The method combines an Affective Knowledge Base (AKB) for semantic grounding with residual-conditioned directional control, optimized through RACO. Experimental results show that, compared with existing baseline methods, AffectRAG achieves better V-A accuracy and content preservation.
Critical strengths
-
The paper addresses continuous affective image editing using target V-A coordinates rather than discrete emotion categories. This is an interesting and meaningful research problem.
-
The proposed framework combines retrieval-based semantic grounding with residual-conditioned directional control. This allows concrete visual attributes to be integrated with residual control for affective image editing.
-
The experiments cover three datasets, multiple baselines, human evaluations, and ablation studies, providing a comprehensive experimental evaluation.
Critical weaknesses
-
The V-A coordinates associated with a visual attribute may not remain stable across different scene contexts. For example, bright red lighting may evoke positive high-arousal emotions in a celebration scene but negative high-arousal emotions in a fire scene. The paper does not adequately explain how the framework determines whether a retrieved attribute's V-A coordinates remain appropriate for a new source image, since the retrieval process does not directly evaluate the attribute's affective effect within the specific image context.
-
The retrieved anchor V-A coordinates are derived from affective associations stored in the knowledge base rather than measurements of the source image after the corresponding semantic edit has been applied. By directly computing the residual as the difference between the target V-A and the retrieved anchor V-A, the method implicitly assumes that the anchor coordinates approximate the actual affective state of the edited image, but this assumption is not directly validated. Consequently, discrepancies between the expected and actual effects of semantic editing may lead to inaccurate residual control.
-
The proposed framework assumes that the retrieved semantic anchor is sufficiently accurate and that the remaining affective residual can be corrected through brightness, saturation, and color-temperature adjustments. However, when the retrieved factor is semantically inappropriate or fails to produce the expected V-A shift, these low-level visual adjustments may be insufficient to correct the error. Since RACO does not explicitly support semantic-level feedback or anchor reselection, it remains unclear how the framework can recover from an incorrect semantic grounding decision.
-
The optimization objective of RACO is not fully aligned with the ultimate goal of continuous affective editing, as its ternary directional reward supervises only the adjustment directions of three visual properties without directly supervising adjustment magnitudes or incorporating the final image's V-A error into the reward. Although Table 6 demonstrates some sensitivity to continuous V-A variations, these results do not clearly establish whether this capability is learned through residual magnitude conditioning or inherited from the pretrained Planner's existing language understanding. More direct experimental evidence is needed to isolate the contribution of magnitude conditioning to continuous affective control.
-
Although the paper provides a mathematical formulation for compatibility filtering, it does not sufficiently explain how the Planner reliably predicts the allowed edit scopes and insertion patterns for a given source image. For example, it remains unclear whether the Planner selects from a predefined metadata vocabulary, how the completeness of its predictions is ensured, and whether these compatibility decisions have been validated against human-annotated ground truth. If the Planner incorrectly excludes a suitable factor, subsequent retrieval cannot recover it, making the reliability of the framework dependent on Qwen3-VL's ability to assess editing feasibility.
-
The validity of the Affective Knowledge Base (AKB) is not sufficiently established through independent human evaluation, as Appendix C indicates that annotators assigned V-A scores to standalone textual descriptions of visual attributes without viewing the corresponding images. The paper does not report inter-rater agreement for these attribute-level annotations or directly verify whether the affective scores assigned to textual descriptions reflect the emotions evoked when the attributes are visually instantiated. Given the potential context dependence of affective perception, additional validation is needed to establish the cross-scene reliability of the AKB and the correspondence between anchor V-A coordinates and the actual V-A of edited images.
-
Although AffectRAG achieves the lowest V-Error and A-Error across all three datasets, the absolute errors remain relatively substantial; for example, the V-Error on EmoSet is 1.434 on a valence scale ranging from 0 to 9. These results demonstrate improved average affective alignment relative to the baselines but do not necessarily establish that the method can precisely reach arbitrary target coordinates in continuous V-A space. The authors should provide a more detailed analysis of error distributions across different target regions, success rates under predefined error thresholds, and the statistical significance of the reported improvements.
Questions for authors
-
Appendix C indicates that the AKB's V-A coordinates were obtained from human ratings of standalone textual attribute descriptions without showing the corresponding images. Have the authors evaluated whether these text-based affective coordinates accurately reflect the perceived V-A of the attributes when instantiated in actual images, particularly across different scene contexts? Additionally, could the authors report the inter-rater agreement for these annotations?
-
The proposed method computes the affective residual as the difference between the target V-A and the retrieved anchor V-A. However, the anchor coordinates are obtained from the AKB rather than from the actual image produced by semantic editing. Have the authors measured how closely the retrieved anchor coordinates correspond to the perceived V-A of the resulting edited images, and how discrepancies affect the accuracy of residual control?
-
How does the framework handle cases where the retrieved semantic factor is inappropriate for the source image or fails to produce the expected affective shift? Since the subsequent residual control primarily adjusts brightness, saturation, and color temperature, can the authors clarify whether the Planner can revise the semantic editing instructions or select an alternative anchor when the initial grounding is incorrect?
-
RACO directly supervises ternary adjustment directions but does not explicitly optimize adjustment magnitudes or the final V-A error. Although Table 6 demonstrates sensitivity to continuous target variations, could the authors provide an ablation that removes residual magnitude information while retaining the same semantic anchors and directional controls? This would help determine whether continuous control is learned through the proposed training objective or primarily inherited from the pretrained Planner.
-
How does the Planner determine the allowed edit scopes and insertion patterns used for compatibility filtering? In particular, does it select from a predefined metadata vocabulary, and have the authors evaluated the accuracy of these predictions against human judgments? Since incorrectly excluding a compatible factor prevents its subsequent retrieval, understanding the reliability of this step is important for assessing the robustness of the framework.
12 comments
Response to Reviewers DvAy on more clarification and validation. (2/5)
Comment 2: Anchor Retrieval
Question 3: How does the framework handle cases where the retrieved semantic factor is inappropriate for the source image or fails to produce the expected affective shift? Since the subsequent residual control primarily adjusts brightness, saturation, and color temperature, can the authors clarify whether the Planner can revise the semantic editing instructions or select an alternative anchor when the initial grounding is incorrect?
Weakness 3: The proposed framework assumes that the retrieved semantic anchor is sufficiently accurate and that the remaining affective residual can be corrected through brightness, saturation, and color-temperature adjustments. However, when the retrieved factor is semantically inappropriate or fails to produce the expected V-A shift, these low-level visual adjustments may be insufficient to correct the error. Since RACO does not explicitly support semantic-level feedback or anchor reselection, it remains unclear how the framework can recover from an incorrect semantic grounding decision.
Authors’ Response:
Thank you for your comment. We will clarify how AffectRAG mitigates incorrect semantic grounding through compatibility filtering, candidate selection, and source-aware instruction planning.
-
Compatibility Filtering and Anchor Selection. As described in Section 3.3, AffectRAG does not directly adopt an arbitrary semantic anchor.
-
The Planner first identifies edit scopes and insertion patterns compatible with the source image, and retrieves only factors sharing the same edit scopes and insertion patterns from AKB, thereby avoiding semantically inappropriate edits.
-
The remaining compatible visual attributes form a candidate pool, ranked by target V–A proximity and editing difficulty. The highest-ranked candidate is selected as the semantic anchor to make the expected affective shift close to the target while remaining easier to achieve.
-
As shown in Table 5, compatibility filtering reduces V/A errors from 1.591/1.765 to 1.568/1.657. Random anchor selection yields higher errors of 1.834/1.907, supporting the effectiveness of candidate filtering and ranking.
-
-
Source-Aware Semantic Planning. The selected anchor provides semantic guidance rather than a fixed editing instruction.
-
The Planner jointly considers the source image, retrieved attributes, target V–A coordinates, and residual to generate context-appropriate semantic edits and visual adjustments.
-
Furthermore, RACO compares multiple candidate plans during training, encouraging more effective integration of semantic attributes and residual controls.
-
These mechanisms help reduce inappropriate semantic grounding, rather than relying solely on low-level visual adjustments.
We have further clarified compatibility filtering, anchor selection, and source-aware semantic grounding in Appendix H [Metadata and Factor Prototypes in Affective Knowledge Base] of the revised manuscript.
Response to Reviewers DvAy on more clarification and validation. (3/5)
Comment 3: Residual Magnitude Conditioning
Question 4: RACO directly supervises ternary adjustment directions but does not explicitly optimize adjustment magnitudes or the final V-A error. Although Table 6 demonstrates sensitivity to continuous target variations, could the authors provide an ablation that removes residual magnitude information while retaining the same semantic anchors and directional controls? This would help determine whether continuous control is learned through the proposed training objective or primarily inherited from the pretrained Planner.
Weakness 4: The optimization objective of RACO is not fully aligned with the ultimate goal of continuous affective editing, as its ternary directional reward supervises only the adjustment directions of three visual properties without directly supervising adjustment magnitudes or incorporating the final image's V-A error into the reward. Although Table 6 demonstrates some sensitivity to continuous V-A variations, these results do not clearly establish whether this capability is learned through residual magnitude conditioning or inherited from the pretrained Planner's existing language understanding. More direct experimental evidence is needed to isolate the contribution of magnitude conditioning to continuous affective control.
Authors’ Response:
Thank you for your valuable suggestion. We provide a controlled ablation to distinguish magnitude conditioning from pretrained Planner capabilities.
-
Controlled Magnitude Ablation. To isolate this contribution, we compare four settings using the same source images, semantic anchors, directional controls, and frozen Executor:
-
w/o Magnitude: Uses the RACO-trained Planner with continuous residual magnitudes removed, while retaining directional information.
-
Shuffled Magnitude: Uses the RACO-trained Planner with magnitudes randomly shuffled within matched anchor-direction groups, preserving their distribution while disrupting correspondence with the targets.
-
Pretrained + Magnitude: Uses the original Qwen3-VL-4B Planner without task-specific RACO training, while providing the correct residual magnitudes. This setting evaluates magnitude-conditioning capabilities inherited from pretraining.
-
RACO + Magnitude: Uses the same Qwen3-VL-4B initialization, further optimized with RACO's directional-control and retrieval-confidence-gated semantic coverage rewards, while providing the same correct residual magnitudes. This setting evaluates the additional contribution of RACO training beyond pretraining.
We follow the original fixed-anchor, same-direction sweeps (Table 7(b)). Semantic anchors are retrieved using the original target V–A coordinates and kept identical across all settings. To isolate magnitude conditioning, we exclude absolute target and anchor coordinates only from the Planner inputs after retrieval, preventing the Planner from reconstructing residual magnitudes.
-
Table R4-3: Controlled Ablation of Residual Magnitude Conditioning.
| Planner | Magnitude Setting | V-Error ↓ | A-Error ↓ | V-Shift Corr. ↑ | A-Shift Corr. ↑ |
|---|---|---|---|---|---|
| RACO-trained | Removed | 0.330 | 0.357 | — | — |
| RACO-trained | Shuffled | 0.314 | 0.343 | 0.120 | 0.091 |
| Pretrained | Correct | 0.307 | 0.332 | 0.419 | 0.395 |
| RACO-trained | Correct | 0.222 | 0.258 | 0.802 | 0.758 |
-
Experimental Findings. The comparisons in Table R4-3 address two key aspects:
-
Magnitude Conditioning: Correct magnitudes improve target accuracy over both magnitude removal and shuffling. The substantially higher shift correlations compared with shuffled magnitudes further indicate that the Planner responds to the actual residual magnitudes rather than arbitrary numerical variations.
-
Contribution Beyond Pretraining: With identical magnitude inputs, RACO improves V/A shift correlations from 0.419/0.395 to 0.802/0.758, indicating that RACO training strengthens magnitude-conditioned responses beyond pretrained Planner capabilities.
-
-
Why we use directional supervision as the training objective instead of directly supervising adjustment magnitudes or the final image's V–A error.
-
Directly optimizing magnitudes or V–A would require incorporating an affective evaluator into reward/loss computation, potentially introducing evaluator bias during optimization.
-
We therefore adopt human-annotated directional supervision and retrieval-confidence-gated semantic rewards, reducing reliance on imperfect evaluator predictions while retaining continuous residual magnitudes for fine-grained instruction generation.
-
These results prove that correct residual magnitude improves continuous affective control, while RACO further enhances the magnitude-conditioned responses.
We have added the ablation and rationale in Appendix F [RACO Optimization Objective] of the revised manuscript.
Response to Reviewers DvAy on more clarification and validation. (1/5)
We would like to express our sincere gratitude to your recognition of our meaningful research, effective design, and comprehensive evaluation. In the following, we we address all your comments with point-by-point responses and relevant revisions in the paper.
Comment 1: Annotation Reliability and Anchor-to-Image Validity
Authors’ Response:
Thank you for your valuable comments. We clarify the AKB annotation protocol, provide experimental evidence of annotation reliability and anchor-to-image validity, and examine how anchor–image discrepancies affect final editing accuracy.
Annotation Protocol and Reliability.
Protocol Clarification:
Annotation Consistency Analysis:
Image-Level and Cross-Context Validation.
Experimental Setup:
Image-Level Validity:
Cross-Context Consistency:
Residual Direction Reliability:
Table R4-1: Validation of AKB Annotation Reliability and Image-Level Affective Validity.
Table R4-2: Impact of Anchor–Image Discrepancies on Target Accuracy.
Together, these additional results support the annotation reliability, anchor–to-image correspondence across different contexts, and the stable effects on editing accuracy.
We have added these analyses to Section 4.4 [Evaluator and Annotation Reliability] of the revised manuscript.