acceptodds
Withdrawn from ICLR 2027

AffectRAG: Continuous Affective Image Editing via Retrieval-Augmented Semantic Grounding and Residual Control

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Affective image editing (AIE) focuses on modulating the emotions evoked by images, yet existing methods typically rely on discrete emotion categories and fail to capture fine-grained affective variations. In this paper, we study continuous affective image editing (CAIE), which aims to edit an image toward a target valence-arousal (V-A) state. Current methods face two key challenges in CAIE. First, V-A values lack explicit visual semantics, making the numeric-to-visual mapping difficult to learn. Second, existing editing mechanisms cannot precisely reach a specified V-A coordinate. To address these challenges, we propose **AffectRAG**, a retrieval-augmented planning framework that combines discrete semantic grounding with continuous residual control. Specifically, to compensate for the lack of visual semantics, we construct a large-scale **Affective Knowledge Base** to ground abstract V-A targets into editable semantic anchors. To further improve editing precision, we introduce **Retrieval-Augmented Residual Control Optimization**, which learns the relationship between anchor-target residuals and visual controls through verifiable rewards and retrieval-confidence gating. Experiments on multiple public affective datasets show that AffectRAG outperforms existing state-of-the-art AIE and general image editing methods in both V-A accuracy and semantic preservation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Affective Image Editing, Valence-Arousal Space, Retrieval-Augmented Generation, Multimodal Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_DvAyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AffectRAG, a continuous affective image editing framework that aims to achieve target Valence-Arousal (V-A) coordinates. The method combines an Affective Knowledge Base (AKB) for semantic grounding with residual-conditioned directional control, optimized through RACO. Experimental results show that, compared with existing baseline methods, AffectRAG achieves better V-A accuracy and content preservation.

Critical strengths
  1. The paper addresses continuous affective image editing using target V-A coordinates rather than discrete emotion categories. This is an interesting and meaningful research problem.

  2. The proposed framework combines retrieval-based semantic grounding with residual-conditioned directional control. This allows concrete visual attributes to be integrated with residual control for affective image editing.

  3. The experiments cover three datasets, multiple baselines, human evaluations, and ablation studies, providing a comprehensive experimental evaluation.

Critical weaknesses
  1. The V-A coordinates associated with a visual attribute may not remain stable across different scene contexts. For example, bright red lighting may evoke positive high-arousal emotions in a celebration scene but negative high-arousal emotions in a fire scene. The paper does not adequately explain how the framework determines whether a retrieved attribute's V-A coordinates remain appropriate for a new source image, since the retrieval process does not directly evaluate the attribute's affective effect within the specific image context.

  2. The retrieved anchor V-A coordinates are derived from affective associations stored in the knowledge base rather than measurements of the source image after the corresponding semantic edit has been applied. By directly computing the residual as the difference between the target V-A and the retrieved anchor V-A, the method implicitly assumes that the anchor coordinates approximate the actual affective state of the edited image, but this assumption is not directly validated. Consequently, discrepancies between the expected and actual effects of semantic editing may lead to inaccurate residual control.

  3. The proposed framework assumes that the retrieved semantic anchor is sufficiently accurate and that the remaining affective residual can be corrected through brightness, saturation, and color-temperature adjustments. However, when the retrieved factor is semantically inappropriate or fails to produce the expected V-A shift, these low-level visual adjustments may be insufficient to correct the error. Since RACO does not explicitly support semantic-level feedback or anchor reselection, it remains unclear how the framework can recover from an incorrect semantic grounding decision.

  4. The optimization objective of RACO is not fully aligned with the ultimate goal of continuous affective editing, as its ternary directional reward supervises only the adjustment directions of three visual properties without directly supervising adjustment magnitudes or incorporating the final image's V-A error into the reward. Although Table 6 demonstrates some sensitivity to continuous V-A variations, these results do not clearly establish whether this capability is learned through residual magnitude conditioning or inherited from the pretrained Planner's existing language understanding. More direct experimental evidence is needed to isolate the contribution of magnitude conditioning to continuous affective control.

  5. Although the paper provides a mathematical formulation for compatibility filtering, it does not sufficiently explain how the Planner reliably predicts the allowed edit scopes and insertion patterns for a given source image. For example, it remains unclear whether the Planner selects from a predefined metadata vocabulary, how the completeness of its predictions is ensured, and whether these compatibility decisions have been validated against human-annotated ground truth. If the Planner incorrectly excludes a suitable factor, subsequent retrieval cannot recover it, making the reliability of the framework dependent on Qwen3-VL's ability to assess editing feasibility.

  6. The validity of the Affective Knowledge Base (AKB) is not sufficiently established through independent human evaluation, as Appendix C indicates that annotators assigned V-A scores to standalone textual descriptions of visual attributes without viewing the corresponding images. The paper does not report inter-rater agreement for these attribute-level annotations or directly verify whether the affective scores assigned to textual descriptions reflect the emotions evoked when the attributes are visually instantiated. Given the potential context dependence of affective perception, additional validation is needed to establish the cross-scene reliability of the AKB and the correspondence between anchor V-A coordinates and the actual V-A of edited images.

  7. Although AffectRAG achieves the lowest V-Error and A-Error across all three datasets, the absolute errors remain relatively substantial; for example, the V-Error on EmoSet is 1.434 on a valence scale ranging from 0 to 9. These results demonstrate improved average affective alignment relative to the baselines but do not necessarily establish that the method can precisely reach arbitrary target coordinates in continuous V-A space. The authors should provide a more detailed analysis of error distributions across different target regions, success rates under predefined error thresholds, and the statistical significance of the reported improvements.

Questions for authors
  1. Appendix C indicates that the AKB's V-A coordinates were obtained from human ratings of standalone textual attribute descriptions without showing the corresponding images. Have the authors evaluated whether these text-based affective coordinates accurately reflect the perceived V-A of the attributes when instantiated in actual images, particularly across different scene contexts? Additionally, could the authors report the inter-rater agreement for these annotations?

  2. The proposed method computes the affective residual as the difference between the target V-A and the retrieved anchor V-A. However, the anchor coordinates are obtained from the AKB rather than from the actual image produced by semantic editing. Have the authors measured how closely the retrieved anchor coordinates correspond to the perceived V-A of the resulting edited images, and how discrepancies affect the accuracy of residual control?

  3. How does the framework handle cases where the retrieved semantic factor is inappropriate for the source image or fails to produce the expected affective shift? Since the subsequent residual control primarily adjusts brightness, saturation, and color temperature, can the authors clarify whether the Planner can revise the semantic editing instructions or select an alternative anchor when the initial grounding is incorrect?

  4. RACO directly supervises ternary adjustment directions but does not explicitly optimize adjustment magnitudes or the final V-A error. Although Table 6 demonstrates sensitivity to continuous target variations, could the authors provide an ablation that removes residual magnitude information while retaining the same semantic anchors and directional controls? This would help determine whether continuous control is learned through the proposed training objective or primarily inherited from the pretrained Planner.

  5. How does the Planner determine the allowed edit scopes and insertion patterns used for compatibility filtering? In particular, does it select from a predefined metadata vocabulary, and have the authors evaluated the accuracy of these predictions against human judgments? Since incorrectly excluding a compatible factor prevents its subsequent retrieval, understanding the reliability of this step is important for assessing the robustness of the framework.

12 comments
AuthorsOpenReview ↗

Response to Reviewers DvAy on more clarification and validation. (1/5)

We would like to express our sincere gratitude to your recognition of our meaningful research, effective design, and comprehensive evaluation. In the following, we we address all your comments with point-by-point responses and relevant revisions in the paper.

Comment 1: Annotation Reliability and Anchor-to-Image Validity

Question 1 & Weakness 6: Appendix C indicates that the AKB's V-A coordinates were obtained from human ratings of standalone textual attribute descriptions without showing the corresponding images. Have the authors evaluated whether these text-based affective coordinates accurately reflect the perceived V-A of the attributes when instantiated in actual images, particularly across different scene contexts? Additionally, could the authors report the inter-rater agreement for these annotations?

Question 2 & Weakness 2: The proposed method computes the affective residual as the difference between the target V-A and the retrieved anchor V-A. However, the anchor coordinates are obtained from the AKB rather than from the actual image produced by semantic editing. Have the authors measured how closely the retrieved anchor coordinates correspond to the perceived V-A of the resulting edited images, and how discrepancies affect the accuracy of residual control?

Authors’ Response:

Thank you for your valuable comments. We clarify the AKB annotation protocol, provide experimental evidence of annotation reliability and anchor-to-image validity, and examine how anchor–image discrepancies affect final editing accuracy.

  • Annotation Protocol and Reliability.

    • Protocol Clarification:

      • In addition to attribute descriptions, AKB annotators were also provided with the affective factors and holistic image abstractions instantiated by each attribute.
      • Therefore, attribute annotations took the corresponding visual content into account.
    • Annotation Consistency Analysis:

      • We further analyze inter-rater agreement and rating dispersion.
      • As shown in Table R4-1, V/A inter-rater agreement reaches 0.87/0.84, with rating stds of 0.62/0.69.
      • These results indicate consistent human affective judgments with relatively limited rating dispersion.
  • Image-Level and Cross-Context Validation.

    • Experimental Setup:

      • We evaluate 100 semantic anchors, each applied to four compatible source images, yielding 400 anchor-only edits in which the attributes are visually instantiated.
    • Image-Level Validity:

      • As shown in Table R4-1, the V/A MAEs between anchor coordinates and emotions evoked in edited images are 0.85/0.94, with Spearman correlations of 0.79/0.75.
      • These results indicate meaningful correspondence between attribute coordinates and image-level perceived affects.
    • Cross-Context Consistency:

      • The within-anchor V/A stds across four contexts are 0.58/0.66, suggesting relatively stable affective effects across different scenes, while allowing for contextual variation.
    • Residual Direction Reliability:

      • We compare the reference target–anchor affect residual with the actual residual between the target and the perceived V–A of edited images.
      • The direction agreement reaches 83.5%/81.0% for V/A, indicating that anchor-derived residuals lead to edits in the expected direction across the evaluated cases.

Table R4-1: Validation of AKB Annotation Reliability and Image-Level Affective Validity.

MetricValenceArousal
AKB Inter-Rater Agreement ↑0.870.84
AKB Rating Std ↓0.620.69
Anchor-to-Image MAE ↓0.850.94
Anchor-to-Image Spearman ↑0.790.75
Cross-Context Std ↓0.580.66
Residual Direction Agreement ↑83.5%81.0%
  • Impact on Final Editing Accuracy.
    • We further divide the 400 instances into low-, medium-, and high-discrepancy groups based on the differences between anchor coordinates and the actual V/A of edited images. Under identical source–target settings, we compare RAG-Only and AffectRAG.
    • As shown in Table R4-2, AffectRAG consistently outperforms RAG-Only across all three groups, indicating that residual-conditioned refinement improves target alignment even when semantic anchors do not precisely match their image-level realization.

Table R4-2: Impact of Anchor–Image Discrepancies on Target Accuracy.

Anchor DiscrepancySamplesRAG-Only Avg. V/A Error ↓AffectRAG Avg. V/A Error ↓
Low2351.461.20
Medium1031.811.51
High622.281.88

Together, these additional results support the annotation reliability, anchor–to-image correspondence across different contexts, and the stable effects on editing accuracy.

We have added these analyses to Section 4.4 [Evaluator and Annotation Reliability] of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewers DvAy on more clarification and validation. (2/5)

Comment 2: Anchor Retrieval

Question 3: How does the framework handle cases where the retrieved semantic factor is inappropriate for the source image or fails to produce the expected affective shift? Since the subsequent residual control primarily adjusts brightness, saturation, and color temperature, can the authors clarify whether the Planner can revise the semantic editing instructions or select an alternative anchor when the initial grounding is incorrect?

Weakness 3: The proposed framework assumes that the retrieved semantic anchor is sufficiently accurate and that the remaining affective residual can be corrected through brightness, saturation, and color-temperature adjustments. However, when the retrieved factor is semantically inappropriate or fails to produce the expected V-A shift, these low-level visual adjustments may be insufficient to correct the error. Since RACO does not explicitly support semantic-level feedback or anchor reselection, it remains unclear how the framework can recover from an incorrect semantic grounding decision.

Authors’ Response:

Thank you for your comment. We will clarify how AffectRAG mitigates incorrect semantic grounding through compatibility filtering, candidate selection, and source-aware instruction planning.

  • Compatibility Filtering and Anchor Selection. As described in Section 3.3, AffectRAG does not directly adopt an arbitrary semantic anchor.

    • The Planner first identifies edit scopes and insertion patterns compatible with the source image, and retrieves only factors sharing the same edit scopes and insertion patterns from AKB, thereby avoiding semantically inappropriate edits.

    • The remaining compatible visual attributes form a candidate pool, ranked by target V–A proximity and editing difficulty. The highest-ranked candidate is selected as the semantic anchor to make the expected affective shift close to the target while remaining easier to achieve.

    • As shown in Table 5, compatibility filtering reduces V/A errors from 1.591/1.765 to 1.568/1.657. Random anchor selection yields higher errors of 1.834/1.907, supporting the effectiveness of candidate filtering and ranking.

  • Source-Aware Semantic Planning. The selected anchor provides semantic guidance rather than a fixed editing instruction.

    • The Planner jointly considers the source image, retrieved attributes, target V–A coordinates, and residual to generate context-appropriate semantic edits and visual adjustments.

    • Furthermore, RACO compares multiple candidate plans during training, encouraging more effective integration of semantic attributes and residual controls.

These mechanisms help reduce inappropriate semantic grounding, rather than relying solely on low-level visual adjustments.

We have further clarified compatibility filtering, anchor selection, and source-aware semantic grounding in Appendix H [Metadata and Factor Prototypes in Affective Knowledge Base] of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewers DvAy on more clarification and validation. (3/5)

Comment 3: Residual Magnitude Conditioning

Question 4: RACO directly supervises ternary adjustment directions but does not explicitly optimize adjustment magnitudes or the final V-A error. Although Table 6 demonstrates sensitivity to continuous target variations, could the authors provide an ablation that removes residual magnitude information while retaining the same semantic anchors and directional controls? This would help determine whether continuous control is learned through the proposed training objective or primarily inherited from the pretrained Planner.

Weakness 4: The optimization objective of RACO is not fully aligned with the ultimate goal of continuous affective editing, as its ternary directional reward supervises only the adjustment directions of three visual properties without directly supervising adjustment magnitudes or incorporating the final image's V-A error into the reward. Although Table 6 demonstrates some sensitivity to continuous V-A variations, these results do not clearly establish whether this capability is learned through residual magnitude conditioning or inherited from the pretrained Planner's existing language understanding. More direct experimental evidence is needed to isolate the contribution of magnitude conditioning to continuous affective control.

Authors’ Response:

Thank you for your valuable suggestion. We provide a controlled ablation to distinguish magnitude conditioning from pretrained Planner capabilities.

  • Controlled Magnitude Ablation. To isolate this contribution, we compare four settings using the same source images, semantic anchors, directional controls, and frozen Executor:

    • w/o Magnitude: Uses the RACO-trained Planner with continuous residual magnitudes removed, while retaining directional information.

    • Shuffled Magnitude: Uses the RACO-trained Planner with magnitudes randomly shuffled within matched anchor-direction groups, preserving their distribution while disrupting correspondence with the targets.

    • Pretrained + Magnitude: Uses the original Qwen3-VL-4B Planner without task-specific RACO training, while providing the correct residual magnitudes. This setting evaluates magnitude-conditioning capabilities inherited from pretraining.

    • RACO + Magnitude: Uses the same Qwen3-VL-4B initialization, further optimized with RACO's directional-control and retrieval-confidence-gated semantic coverage rewards, while providing the same correct residual magnitudes. This setting evaluates the additional contribution of RACO training beyond pretraining.

    We follow the original fixed-anchor, same-direction sweeps (Table 7(b)). Semantic anchors are retrieved using the original target V–A coordinates and kept identical across all settings. To isolate magnitude conditioning, we exclude absolute target and anchor coordinates only from the Planner inputs after retrieval, preventing the Planner from reconstructing residual magnitudes.

Table R4-3: Controlled Ablation of Residual Magnitude Conditioning.

PlannerMagnitude SettingV-Error ↓A-Error ↓V-Shift Corr. ↑A-Shift Corr. ↑
RACO-trainedRemoved0.3300.357——
RACO-trainedShuffled0.3140.3430.1200.091
PretrainedCorrect0.3070.3320.4190.395
RACO-trainedCorrect0.2220.2580.8020.758
  • Experimental Findings. The comparisons in Table R4-3 address two key aspects:

    • Magnitude Conditioning: Correct magnitudes improve target accuracy over both magnitude removal and shuffling. The substantially higher shift correlations compared with shuffled magnitudes further indicate that the Planner responds to the actual residual magnitudes rather than arbitrary numerical variations.

    • Contribution Beyond Pretraining: With identical magnitude inputs, RACO improves V/A shift correlations from 0.419/0.395 to 0.802/0.758, indicating that RACO training strengthens magnitude-conditioned responses beyond pretrained Planner capabilities.

  • Why we use directional supervision as the training objective instead of directly supervising adjustment magnitudes or the final image's V–A error.

    • Directly optimizing magnitudes or V–A would require incorporating an affective evaluator into reward/loss computation, potentially introducing evaluator bias during optimization.

    • We therefore adopt human-annotated directional supervision and retrieval-confidence-gated semantic rewards, reducing reliance on imperfect evaluator predictions while retaining continuous residual magnitudes for fine-grained instruction generation.

These results prove that correct residual magnitude improves continuous affective control, while RACO further enhances the magnitude-conditioned responses.

We have added the ablation and rationale in Appendix F [RACO Optimization Objective] of the revised manuscript.

Reviewer_heHgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies continuous affective image editing (CAIE) and proposed Retrieval-Augmented Residual Control Optimization to achieve better V-A accuracy and semantic preservation.

Critical strengths

The proposed technique achieves better V-A accuracy and semantic preservation.

Critical weaknesses

There is no standardize benchmarking.

Questions for authors

How do you ensure the fairness of human judgement in the evaluation.

2 comments
AuthorsOpenReview ↗

Response to Reviewers heHg on standardize benchmarking and human evaluation.

Thank you very much for your positive assessment of our work.

Your constructive comments are exceedingly helpful to improve our paper. We have carefully incorporated them in the revised paper. In the following, your comments are first stated and then followed by our point-by-point responses.

Question 1:

How do you ensure the fairness of human judgement in the evaluation.

Authors’ Response:

Thank you for your question. Our human evaluation (Section 4.2) adopts the following measures to ensure fairness and minimize potential biases:

  • Balanced Sampling: All eight methods are evaluated on the same 120 source–target pairs, with 40 pairs per dataset and 10 per V–A quadrant, reducing sampling bias across datasets and affective regions.

  • Blind and Randomized Evaluation: 30 participants assess outputs with method identities concealed and presentation order randomized, preventing method identity disclosure.

  • Uniform Rating and Aggregation: All methods use identical rating scales. Ratings are first averaged per output and then across evaluation pairs, ensuring consistent scoring and equal weighting of samples.

  • Evaluator–Human Agreement:

    • Our existing User Study (Table 3) shows method-level agreement between automatic and human evaluations.
    • To further assess the fairness of human judgement, we compare human ratings with the V–A predictions of the CLIP-based evaluator on the source and edited images in the user study.
    • As shown in Table R3-1, the V/A MAEs between human ratings and automatic predictions are 0.72/0.78 for source images and 0.85/0.93 for edited images. The corresponding Spearman correlations are 0.84/0.80 and 0.79/0.75.
    • These results indicate substantial evaluator–human agreement at the individual-image level.

Table R3-1: Evaluator–Human Agreement Analysis.

EvaluationMetricValenceArousal
Source ImagesMAE ↓0.720.78
Spearman ↑0.840.80
Edited ImagesMAE ↓0.850.93
Spearman ↑0.790.75

We have added the evaluator–human agreement analysis to Section 4.4 [Evaluator and Annotation Reliability] and further detailed the validation protocol in Appendix B [Models and Baseline Methods] of the revised manuscript.


Weakness 1:

There is no standardize benchmarking.

Authors’ Response:

Thank you for this comment. We would like to clarify our unified benchmarking protocol (Section 4.1), which ensures consistent evaluation across datasets and competing methods.

  • Unified Benchmarking Settings:
    • We evaluate all methods on three public datasets (EmoSet, EMOTIC, and FindingEmo) using identical test images and target V–A coordinates.
    • All V–A annotations are normalized to [0, 9], and the same four evaluation metrics (V-Error, A-Error, CLIP-I, and LPIPS) are applied.
  • Consistent Baseline Comparisons:
    • We compare AffectRAG against seven representative baselines under the same evaluation protocol, using the same frozen CLIP-based affective evaluator.
    • Model configurations and prompt templates are documented in Appendices B and E.
  • Under this unified benchmarking protocol, Table 1 shows that AffectRAG achieves lower V/A errors across all three datasets.

We have further clarified the unified benchmarking protocol in Appendix B [Models and Baseline Methods] of the revised manuscript.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer heHg,

We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.

Could you please kindly let us know if our rebuttal has addressed your concerns?

If you have any further questions regarding our work, we would be delighted to address them.

If our responses have addressed your concerns, we would greatly appreciate it if you would consider whether the paper merits a higher overall rating.

Best regards,

Authors

Reviewer_s6gdOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

AffectRAG addresses continuous affective image editing toward specified valence–arousal coordinates. It retrieves compatible semantic anchors from an affective knowledge base and uses a vision-language planner to generate residual-conditioned editing instructions for a frozen image editor. The planner is optimized with directional-control rewards and retrieval-coverage rewards weighted by retrieval confidence. Experiments on three datasets, human evaluation, and local target sweeps indicate improved affective alignment and content preservation.

Critical strengths
  1. The decomposition into semantic anchoring and residual visual adjustment provides an intuitive approach to translating abstract affective coordinates into executable edits. The compatibility filtering and retrieval-confidence weighting address concrete difficulties in using retrieved semantic content.

  2. The evaluation combines automatic metrics with human judgments and controlled target sweeps. Holding the semantic anchor fixed is particularly useful for testing whether the method responds to finer target changes beyond simply retrieving different emotional elements.

Critical weaknesses
  1. The comparison does not fully separate the proposed planning method from stronger prompting or planning already available to general-purpose editors. The internal processing of closed-source systems such as Seedream is not established in the paper, so it is unclear whether the comparison gives each system an equally effective interface to the numeric affective target. The existing same-executor ablations are useful, but do not establish the benefit over a carefully prompted general-purpose planner or show whether the proposed planner transfers across executors. Could the authors compare direct editing, a generic planning/prompt-refinement stage, and AffectRAG using the same executor? Applying these variants to an additional capable editor would also clarify whether the added planner consistently helps or sometimes overconstrains otherwise useful editing behavior and source-content preservation.

  2. The reliability of the affective annotations and automatic evaluator needs more direct validation. EmoSet’s added labels and the knowledge-base coordinates are human annotations, while automatic evaluation uses a separate frozen CLIP-based predictor. The paper should establish the reliability of both components. In particular, knowledge-base attributes are rated as isolated text without their original images, which may not reflect their affective effect when inserted into a new scene. Could the authors report inter-annotator agreement and rating dispersion, alongside the evaluator’s error and correlation with independent human ratings on held-out source and edited images? The user study supports the overall ranking, but does not by itself establish the evaluator’s sensitivity to small continuous changes.

Questions for authors

See Weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewers s6gd on planning comparison and annotation reliability. (1/2)

We sincerely appreciate your recognition of our methodological design and comprehensive evaluation. Below, we address your constructive comments with detailed point-by-point responses and relevant revisions in the paper.

Comment 1: Planning Comparison and Cross-Executor Generalization

The comparison does not fully separate the proposed planning method from stronger prompting or planning already available to general-purpose editors. The internal processing of closed-source systems such as Seedream is not established in the paper, so it is unclear whether the comparison gives each system an equally effective interface to the numeric affective target. The existing same-executor ablations are useful, but do not establish the benefit over a carefully prompted general-purpose planner or show whether the proposed planner transfers across executors. Could the authors compare direct editing, a generic planning/prompt-refinement stage, and AffectRAG using the same executor? Applying these variants to an additional capable editor would also clarify whether the added planner consistently helps or sometimes overconstrains otherwise useful editing behavior and source-content preservation.

Authors’ Response:

Thank you for your comment. We conduct controlled experiments to assess the benefits of AffectRAG over direct editing, generic planning, and an additional editor.

  • Planner Ablation.

    • We compare Direct Editing, Generic Planner, and AffectRAG under identical frozen Executors.
    • The Generic Planner translates V–A targets into editing instructions without AKB retrieval or residual control.
    • Both trained Planners share the same Qwen3-VL-4B backbone, training data, initialization, and optimization budget.
  • Cross-Executor Evaluation. We apply the above variants on FLUX.2-klein-9B and Qwen-Image-Edit.

Table R2-1: Planner Comparison Across Executors.

ExecutorMethodV-Error ↓A-Error ↓CLIP-I ↑LPIPS ↓
FLUX.2-klein-9BDirect Editing2.0612.0920.8610.612
Generic Planner1.7441.8210.8700.506
AffectRAG1.5681.6570.8790.381
Qwen-Image-EditDirect Editing1.8831.9160.8600.657
Generic Planner1.7381.7850.8680.541
AffectRAG1.5521.6100.8770.402
  • Results.
    • As illustrated in Table R2-1, Generic Planner improves upon Direct Editing, while AffectRAG further reduces V/A errors by 10.1%/9.0% and 10.7%/9.8% on the two Executors.
    • AffectRAG also achieves higher CLIP-I and lower LPIPS, indicating better source-content preservation.

These results support the effectiveness of retrieval-anchored residual planning beyond generic prompt refinement, with consistent improvements across Executors.

We have added the matched planner–executor analysis to Section 4.3 [Ablation Studies] of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewers s6gd on planning comparison and annotation reliability. (2/2)

Comment 2: Annotation Reliability.

The reliability of the affective annotations and automatic evaluator needs more direct validation. EmoSet’s added labels and the knowledge-base coordinates are human annotations, while automatic evaluation uses a separate frozen CLIP-based predictor. The paper should establish the reliability of both components. In particular, knowledge-base attributes are rated as isolated text without their original images, which may not reflect their affective effect when inserted into a new scene. Could the authors report inter-annotator agreement and rating dispersion, alongside the evaluator’s error and correlation with independent human ratings on held-out source and edited images? The user study supports the overall ranking, but does not by itself establish the evaluator’s sensitivity to small continuous changes.

Authors’ Response:

Thank you for your comment. We would like to clarify the attribute annotation protocol and provide additional evidence to assess the reliability of affective annotations and automatic evaluation.

  • Annotation Reliability and Attribute Validity.
    • Annotation Protocol:

      • AKB annotators were provided with attribute descriptions, the affective factors into which these attributes were inserted, and holistic image abstractions.
      • Thus, the annotations considered the affective effects of attributes when inserted into scene contexts.
    • Annotation Consistency Analysis:

      • We measure inter-annotator agreement using the intraclass correlation coefficient (ICC). We also compute within-item rating standard deviations (Std).
      • As shown in Table R2-2, V/A agreement reaches 0.87/0.84 for AKB attributes and 0.89/0.85 for EmoSet source images. The rating stds are 0.62/0.69 and 0.59/0.66.
      • These results indicate high agreement and limited rating dispersion among annotators.
    • Image-Level Validation:

      • We select 100 semantic anchors and insert each anchor's visual attributes into four compatible source-image scenes. This produces 400 anchor-only edited images, without residual refinement. We then compare the anchor coordinates with the affective scores of these edited images.
      • As shown in Table R2-2, the V/A MAEs are 0.85/0.94, with Spearman correlations of 0.79/0.75. The within-anchor cross-context standard deviations are 0.58/0.66.
      • These results indicate meaningful image-level correspondence and consistent affective effects across different scenes.

Table R2-2: Validation of Annotation Reliability and Anchor-to-Image Consistency.

MetricValenceArousal
AKB Inter-Annotator Agreement ↑0.870.84
AKB Rating Std ↓0.620.69
EmoSet Inter-Annotator Agreement ↑0.890.85
EmoSet Rating Std ↓0.590.66
Anchor-to-Image MAE ↓0.850.94
Anchor-to-Image Spearman ↑0.790.75
Cross-Context Std ↓0.580.66
  • Evaluator–Human Agreement and Fine-Grained Sensitivity.
    • Evaluator–Human Agreement:

      • Our existing 30-participant User Study (Table 3) shows method-level agreement between automatic and human evaluations.
      • To further assess the reliability of the evaluator, we compare its predictions with independent human V–A ratings on held-out source and edited images.
      • As shown in Table R2-3, the evaluator achieves V/A MAEs of 0.72/0.78 for source images and 0.85/0.93 for edited images. The corresponding Spearman correlations are 0.84/0.80 and 0.79/0.75.
      • These results indicate substantial agreement with human perception at the individual-image level.
    • Fine-Grained Sensitivity Analysis:

      • We further collect independent human ratings for the fixed-anchor V–A sweeps (Table 7), using target increments of 0.2. We compare evaluator-predicted affective shifts with human-perceived changes.
      • As shown in Table R2-3, the V/A shift correlations reach 0.82/0.78, with adjacent-pair directional agreement of 85.3%/82.4%.
      • These results indicate that the evaluator captures fine-grained affective variations consistent with human perception, beyond method-level agreement.

Table R2-3: Evaluator–Human Agreement and Fine-Grained Sensitivity Analysis.

EvaluationMetricValenceArousal
Source ImagesMAE ↓0.720.78
Spearman ↑0.840.80
Edited ImagesMAE ↓0.850.93
Spearman ↑0.790.75
Fine-Grained SweepsShift Correlation ↑0.820.78
Direction Agreement ↑85.3%82.4%

We have added these analyses to Section 4.4 [Evaluator and Annotation Reliability] of the revised manuscript.


We have made every effort to address the reviewers’ concerns with additional experiments and analyses, and sincerely hope the reviewer can reconsider a positive overall score based on these revisions.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer s6gd,

We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.

Could you please kindly let us know if our rebuttal has addressed your concerns?

If you have any further questions regarding our work, we would be delighted to address them.

If our responses have addressed your concerns, we would greatly appreciate your reconsideration of whether the paper merits a positive overall recommendation.

Best regards,

Authors

Reviewer_4XxeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work focuses on continuous affective image editing, which aims to modify an image toward a target valence–arousal coordinate while preserving its main semantic content. Specifically, the authors propose AffectRAG, which combines retrieval-based semantic grounding with residual-conditioned visual control. In AffectRAG, the core idea is to ground numerical affective targets in retrieved visual concepts, then refine the result through residual-guided lighting and color adjustments. To this end, the authors construct an Affective Knowledge Base and train a vision-language planner using directional-control and retrieval-confidence-gated semantic coverage rewards. The planner generates editing instructions for a frozen image editor. The experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  1. In general, the manuscript is well written and easy to follow.

  2. This work evaluates the method on three datasets and includes a human study. The consistent improvements in automatic and human evaluations provide evidence of practical effectiveness.

  3. The fixed-anchor target sweeps are useful for examining sensitivity to small changes in the requested affective coordinates, beyond comparisons between discrete emotion categories.

Critical weaknesses
  1. The connection between the proposed residual and the required image-level correction is insufficiently established. Specifically, anchor coordinates are derived from human ratings of standalone textual attribute descriptions, whereas the affect evoked by an edited image depends on the source content and how the retrieved attributes are incorporated. Therefore, the target–anchor difference may not accurately represent the correction needed to reach the desired image-level affect.

  2. In the proposed method, RACO primarily rewards agreement with ternary control labels and the inclusion of retrieved semantic expressions in the generated instruction. It does not directly optimize adjustment magnitude or the final image’s distance from the target. Although Table 6 supports local sensitivity and monotonicity, these properties do not by themselves establish accurate calibration across the continuous target space. Overall, the method appears to offer a useful combination of retrieval, instruction planning, and reward optimization, but the evidence for a distinct technical advance in precise continuous affective control is not sufficiently convincing.

  3. The comparison with general editing baselines may partly reflect the benefit of an added planning stage. AffectRAG converts numerical targets into visual instructions, while these baselines receive numerical targets directly (see Appendix E). Table 3 partially addresses this issue, but a matched planner–executor comparison would strengthen the evaluation.

Questions for authors
  1. How well do the text-derived anchor coordinates predict the effect of images edited using those anchors? Does this relationship remain consistent across different source-image contexts?

  2. Does the method maintain target accuracy when crossing anchor-selection boundaries or varying both V and A simultaneously? The fixed-anchor, single-axis sweeps do not address these cases.

  3. Have you compared AffectRAG with the same planner–executor pipeline without retrieval and residual control, using matched training conditions? This would help distinguish the gains from the proposed components from those of the added planning stage.

4 comments
AuthorsOpenReview ↗

Response to Reviewers 4Xxe on affective anchor validity, continuous control, and planner comparisons. (1/3)

Thank you very much for recognizing our manuscript consistent improvements and comprehensive evaluation. In the following, we provide our point-by-point responses to your comments and describe the corresponding revisions in our paper.

Comment 1: Anchor-to-Image Validity

Question 1: How well do the text-derived anchor coordinates predict the effect of images edited using those anchors? Does this relationship remain consistent across different source-image contexts?

Weakness 1: The connection between the proposed residual and the required image-level correction is insufficiently established. Specifically, anchor coordinates are derived from human ratings of standalone textual attribute descriptions, whereas the affect evoked by an edited image depends on the source content and how the retrieved attributes are incorporated. Therefore, the target–anchor difference may not accurately represent the correction needed to reach the desired image-level affect.

Authors’ Response:

Thank you for your comment. We would like to clarify the annotation protocol and provide experimental evidence for anchor-to-image affective validity.

  • Clarification of anchor annotations.

    • Anchor coordinates are not annotated based solely on attribute descriptions. Annotators were also provided with the affective factors into which the attributes were incorporated, as well as holistic image abstractions reflecting the corresponding visual context. Therefore, anchor annotations account for how attributes are incorporated into visual content, rather than treating them as standalone textual descriptions.
    • AKB coordinates serve as semantic affective references instead of exact image-level predictions. The target–anchor residual provides reference-conditioned guidance, while the Planner jointly considers the source image, retrieved attributes, target coordinates, and residual to generate editing instructions.
  • Anchor-to-image affective consistency. We evaluate 100 semantic anchors, each applied to four compatible source images with different affective contexts (400 instances). For each pair, we generate an anchor-only edit without residual refinement and estimate the V–A coordinates.

As shown in Table R1-1, we make three observations:

  1. Image-level affective correspondence. We compute V/A MAEs and Spearman correlations between the annotated anchor coordinates and the affective scores evoked by their corresponding anchor-only edited images. The results indicate that text-derived anchor coordinates can predict the affect of images edited using those anchors to a meaningful extent.

  2. Cross-context consistency. The relatively small within-anchor standard deviations of the edited-image V–A scores indicate a consistent relationship between anchor coordinates and edited affect across different source-image contexts.

  3. Residual direction reliability. We compare the reference affect residual between the target and the anchor with the image-grounded affect correction between the target and the anchor-only edited image. The direction agreement reaches 83.5%/81.0% for V/A, indicating that anchor-derived residuals provide consistent correction directions in most evaluated cases.

These results support the use of semantic anchors as affective references for residual control. The existing Tables 4 and 7 further demonstrate the improvements in target accuracy and continuous affective sensitivity achieved with semantic anchoring.

We have incorporated the new anchor-to-image validity analysis into Section 4.4 [Evaluator and Annotation Reliability] and provided further clarifications on AKB annotations and anchor validity in Appendix C [Annotation and Data Preparation] of the revised manuscript.

Table R1-1: Evaluation of Anchor-to-Image Affective Validity.

MetricValenceArousal
Anchor-to-Image MAE ↓0.850.94
Spearman Correlation ↑0.790.75
Cross-Context Std ↓0.580.66
Residual Direction Agreement ↑83.5%81.0%
AuthorsOpenReview ↗

Response to Reviewers 4Xxe on affective anchor validity, continuous control, and planner comparisons. (2/3)

Comment 2: Precision of Continuous Affective Control

Question 2: Does the method maintain target accuracy when crossing anchor-selection boundaries or varying both V and A simultaneously? The fixed-anchor, single-axis sweeps do not address these cases.

Weakness 2: In the proposed method, RACO primarily rewards agreement with ternary control labels and the inclusion of retrieved semantic expressions in the generated instruction. It does not directly optimize adjustment magnitude or the final image’s distance from the target. Although Table 6 supports local sensitivity and monotonicity, these properties do not by themselves establish accurate calibration across the continuous target space. Overall, the method appears to offer a useful combination of retrieval, instruction planning, and reward optimization, but the evidence for a distinct technical advance in precise continuous affective control is not sufficiently convincing.

Authors’ Response:

Thank you for the comment. We would like to clarify our continuous control mechanism and provide additional experimental evidence.

  • Continuous control beyond direction supervision.

    • Although RACO adopts ternary directional supervision, the Planner also receives continuous residual magnitudes, which condition the strength and spatial scope of generated editing instructions.
    • As shown in Table 7(b), AffectRAG achieves V/A shift correlations of 0.819/0.776, compared with 0.143/0.119 for RAG-Only, demonstrating sensitivity to continuous residual magnitudes even when the predicted direction vectors remain unchanged.
  • Why we use directional supervision instead of continuous V–A supervision.

    • Directly optimizing image-level V–A errors would require incorporating an affective evaluator into reward/loss computation, potentially introducing evaluator bias during optimization.
    • We therefore adopt human-annotated directional supervision and retrieval-confidence-gated semantic rewards, reducing reliance on imperfect evaluator predictions while retaining continuous residual magnitudes for fine-grained instruction generation.
  • Joint V–A control across anchor boundaries.

    • We conduct simultaneously varying 2D V–A target sweeps on 30 held-out source images (10 per dataset). For each image, we locate an anchor-selection boundary through a coarse retrieval scan over [1, 8] with a step size of 0.5, selecting the boundary closest to the V–A center.
    • We construct a 5×5 target grid centered on this boundary, varying both V and A over {-1, -0.5, 0, 0.5, 1}. Each target undergoes normal anchor retrieval.
    • We evaluate all local target pairs with simultaneous V–A variations. These pairs cover same-direction and opposite-direction changes with equal and unequal magnitudes. Each pair is categorized as Within-Anchor (same Top-1 anchor) or Cross-Anchor (different Top-1 anchors).
    • We compare AffectRAG with RAG-Only under identical source images, targets, and retrieval settings, reporting Avg. V/A Error and Transition Error, defined as the L1 discrepancy between achieved and requested affective shifts.

The results are presented in Table R1-2. AffectRAG achieves lower target and transition errors in both settings. Under Cross-Anchor transitions, AffectRAG reduces average V/A error from 1.84 to 1.40 and transition error from 1.13 to 0.63. Its target error remains close to the Within-Anchor setting (1.40 vs. 1.30).

Together with Table 7(b), these findings provide further evidence that AffectRAG supports fine-grained continuous affective control beyond direction supervision, including simultaneous V–A variations and anchor transitions.

We have extended the continuous control analysis to include simultaneous V–A variations and anchor transitions in Appendix D [Continuous Control and Target Accuracy Analyses]. Further details on residual magnitude conditioning and directional supervision are provided in Appendix F [RACO Optimization Objective] of the revised manuscript.

Table R1-2: Simultaneously Varying V–A Control Across Anchor-Selection Boundaries.

SettingMethodAvg. V/A Error ↓Transition Error ↓
Within-AnchorRAG-Only1.600.75
Within-AnchorAffectRAG1.300.52
Cross-AnchorRAG-Only1.841.13
Cross-AnchorAffectRAG1.400.63
AuthorsOpenReview ↗

Response to Reviewers 4Xxe on affective anchor validity, continuous control, and planner comparisons. (3/3)

Comment 3: Matched Planner–Executor Comparison

Question 3: Have you compared AffectRAG with the same planner–executor pipeline without retrieval and residual control, using matched training conditions? This would help distinguish the gains from the proposed components from those of the added planning stage.

Weakness 3: The comparison with general editing baselines may partly reflect the benefit of an added planning stage. AffectRAG converts numerical targets into visual instructions, while these baselines receive numerical targets directly (see Appendix E). Table 3 partially addresses this issue, but a matched planner–executor comparison would strengthen the evaluation.

Authors’ Response:

Thank you for your suggestion. We provide a matched comparison to isolate the benefits of retrieval-anchored residual planning.

  • Experimental Setup.

    • We compare Direct Editing, Generic Planner (without retrieval or residual control), and AffectRAG using the same frozen Executor.
    • Both trained Planners share the Qwen3-VL-4B backbone, training data, initialization, and optimization budget.
    • We additionally apply all variants on Qwen-Image-Edit for cross-executor evaluation.
  • Results. As shown in Table R1-3, AffectRAG outperforms Generic Planner on both Executors, reducing V/A errors by 10.1%/9.0% on FLUX.2-klein-9B and 10.7%/9.8% on Qwen-Image-Edit, while achieving higher content preservation.

Table R1-3: Matched Planner Comparison Across Executors.

ExecutorMethodV-Error ↓A-Error ↓CLIP-I ↑LPIPS ↓
FLUX.2-klein-9BDirect Editing2.0612.0920.8610.612
Generic Planner1.7441.8210.8700.506
AffectRAG1.5681.6570.8790.381
Qwen-Image-EditDirect Editing1.8831.9160.8600.657
Generic Planner1.7381.7850.8680.541
AffectRAG1.5521.6100.8770.402

These results demonstrate that AffectRAG's improvements extend beyond generic planning and generalize across Executors.

We have incorporated the matched planner–executor comparison into Section 4.3 [Ablation Studies] of the revised manuscript.


We have carefully addressed all concerns with supplementary experiments and analyses, and sincerely hope the reviewer can reconsider the overall score in light of these revisions.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer 4Xxe,

We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.

Could you please kindly let us know if our rebuttal has addressed your concerns?

If you have any further questions regarding our work, we would be delighted to address them.

If our responses have addressed your concerns, we would greatly appreciate your reconsideration of your overall assessment.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.