acceptodds
Under review as a conference paper at ICLR 2027

Same Story, Another Culture: Narrative Cultural Re-instantiation for Cross-Cultural Video Generation

Abstract

Reinterpreting a narrative across cultures requires more than changing visual appearance: cultural substitutions must jointly preserve narrative functions and relations. We introduce Narrative Cultural Re-instantiation, the task of generating a target-cultural video from a source-cultural narrative while preserving its specified story structure. The central challenge is that individually plausible substitutions need not form a coherent whole. We propose CultureWeaver, a training-free framework that separates cultural correspondence from visual realization. An evidence-grounded Visual Cultural Memory supplies visually supported candidates annotated with narrative functions, affordances, and relations. CultureWeaver organizes retrieved candidates into a graph and jointly aligns them with the narrative through relation-aware semi-relaxed structural transport, balancing functional compatibility and relational consistency while allowing irrelevant candidates to receive negligible mass. Frozen generative models render the resulting cultural bindings into videos. We construct a benchmark of 200 narrative scenes across two target-cultural settings, with task-specific evaluation of narrative preservation, cultural fidelity, functional validity, and relational coherence. Automated evaluation and human judgments favor CultureWeaver over competing methods, while controlled ablations support the benefits of structured cultural memory and joint correspondence. Code and benchmark data are available in our https://anonymous.4open.science/r/ICLR2027_CultureWeaver-B317/anonymous repository.

Primary area:
generative models
Keywords:
Narrative Cultural Re-instantiation, Cross-Cultural Video Generation, Structural Correspondence, Narrative Preservation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_R74VOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Narrative Cultural Re-instantiation, a task that aims to generate a video that reinterprets a source narrative within a specified target culture while preserving its narrative structure. To address this task, the authors propose CultureWeaver, a training-free framework that leverages a Visual Cultural Memory (VCM) constructed from documented cultural sources. The authors instantiate this memory for two historical cultural contexts: Chinese and Persianate. Each VCM is represented as a graph of cultural records (characters, objects, and scenes), their supported relations, and associated visual evidence.

The paper also introduces four task-specific evaluation metrics covering narrative preservation, cultural content fidelity, functional validity, and relational coherence. CultureWeaver is compared against several baselines using six evaluation metrics, with both automatic and human evaluations demonstrating consistent improvements in narrative and cultural fidelity. Ablation studies investigate the contributions of the cultural memory representation and structural correspondence mechanism.

Critical strengths

The paper presents comprehensive experiments, including both automatic and human evaluations across two target cultures, as well as ablations of the proposed method's components. CultureWeaver consistently outperforms the baselines on the main narrative and cultural evaluation metrics, while the ablations demonstrate the contributions of most individual components. Overall, the experimental results support the effectiveness of the proposed methodology, subject to the clarifications requested in the "Questions for Authors" section.

Critical weaknesses

Limited robustness to cultural-memory coverage and unclear interplay with parametric knowledge.

CultureWeaver relies on a cultural memory whose records and relations may not fully capture broader cultural compatibility. Graph connectivity may reflect co-occurrence in source illustrations rather than meaningful cultural relationships, and potentially favor suboptimal substitutions. Moreover, the method requires assigning every narrative node to a memory candidate, even when no suitable match exists. The experiments do not assess CultureWeaver's performance in such scenarios, and it remains unclear whether the method would still outperform baselines relying primarily on parametric cultural knowledge.

On a relevant note, while the parametric-only ablation in Table 4 demonstrates the benefits of external grounding, it does not disentangle conceptual cultural knowledge (e.g., social roles and relationships) from visual knowledge (e.g., clothing and architecture). It remains unclear when the method should rely on the external graph versus the models' potentially broader parametric knowledge, particularly when the two provide conflicting or incomplete evidence. The paper would benefit from evaluating robustness under limited memory coverage and examining how these two knowledge sources complement each other.

Questions for authors
  1. Ablation with retrieved images only: Could the authors clarify whether the flat visual memory baseline in Table 4 uses the same retrieved images as the full CultureWeaver method, but without access to the graph's structured cultural information? If not, an ablation that provides the same retrieved images to the generation pipeline, without the graph's semantic and relational information, would help disentangle the contributions of visual grounding and structured cultural knowledge.

  2. Instance-level agreement between human and automated evaluations: Table 3 reports agreement between human and VLM-based evaluations using method-level rankings aggregated across instances. Could the authors also report instance-level ranking agreement (e.g., Spearman or Kendall correlation per scene and target culture)? High method-level agreement does not necessarily imply that the automated metrics reliably distinguish the quality of individual generations. Since both human rankings and automated scores are already available, this analysis would provide stronger validation of the proposed evaluation metrics.

  3. Prompts for cultural-memory construction: The supplementary code includes prompts used in the generation and evaluation pipelines, but I could not find those used to construct the Visual Cultural Memory. Could the authors provide the prompts used to extract and annotate cultural records and relationships, or clarify where these can be found? These are important for reproducibility, given the central role of memory construction in the proposed methodology.

  4. Ethical considerations: Could the authors expand on the ethical risks associated with the proposed task and methodology, beyond acknowledging the work's research-oriented nature? Discussing concrete risks and possible mitigation strategies would strengthen the ethical considerations section, which is limited at the moment.

Typo: Line 135: openration -> operation

5 comments

Comment 1/3 — Critical Weaknesses: Cultural Relations, Memory Coverage, and Parametric Knowledge

Thank you for your thoughtful review and constructive suggestions. We have added controlled experiments and further clarified the roles of cultural evidence and structural correspondence.

1. Are VCM relations merely image co-occurrence?

No. VCM encodes functions, affordances, provenance, and semantically labeled directed relations (Secs. 4.2–4.3; Apps. J.4, L.2). Same-image assignment is not enforced: missing relations incur a finite cost of 0.90, and Full bindings span 3.04 source units on average (Apps. I.1, M.5).

Removing relation semantics reduces CRC from 88.5 to 78.4 (Table 4(B)). Removing context-derived edges while fixing candidates and evidence-linked relations decreases CRC by 2.93 [1.27, 4.55] (Table 23, 95% CI). App. D (Tables 7–8) further shows a relation-supported assignment reducing joint cost by 5.9%.

These results support the contribution of directed relational evidence beyond simple image co-occurrence.

2. How does CultureWeaver handle incomplete memory?

CultureWeaver preserves narrative roles, functions, and directed interactions, selecting the best-supported functional correspondences from available VCM candidates without requiring literal cultural equivalents (Sec. 4.3; App. O.2).

Our new coverage experiments (App. O.1, Table 28) evaluate 50 stories across two cultures. Removing essential counterpart families decreases CRV/CRC by 2.85/5.34; retaining 50% of nodes decreases them by 2.90/4.44; retaining 50% of edges decreases CRC by 4.06.

These results show that functional coverage supports suitable substitutions, while relational coverage supports coherent interactions.

3. Does external grounding improve over parametric-only generation?

Yes, under complete-memory conditions. Table 4(A) reports CRC 88.5 for Full versus 79.2 for Parametric-only. Table 28 further measures performance changes as cultural evidence becomes sparser.

Together, these experiments demonstrate the benefit of external grounding and characterize its dependence on memory coverage, without assuming superiority at every sparsity level.

4. What do conceptual cultural knowledge and visual evidence contribute separately?

Cultural functions, affordances, and relations guide which records satisfy narrative requirements; linked images guide their visual realization (Sec. 4.3; App. L.4).

Table 4(A) compares Full with Flat Visual Memory using the same underlying archive (CRC 88.5 vs. 80.4). More directly, App. M.2 (Tables 19–20) fixes the same six images across 100 conditions, using identical GPT-4o first-frame and PixVerse V6 video backbones.

MethodCRVCRC
Images-only84.1079.70
Independent Matching85.3082.80
CultureWeaver (Full)86.1088.20

Images-only uses references without explicit cultural bindings; Independent adds node-wise bindings; Full adds RA-srFGW joint binding and directed relational conditioning.

Full improves CRC over Images-only by +8.50 [4.63, 12.42] and over Independent by +5.40 [0.79, 10.24] (95% paired CIs). Both intervals exclude zero, while the CRV differences are not statistically resolved.

Thus, structured cultural conditioning improves relational coherence even with identical images, complementing structure-guided reference selection.

5. When VCM evidence and parametric knowledge disagree, which governs assignment?

RA-srFGW determines explicit bindings using VCM-grounded functional and relational costs. Frozen models interpret narratives, plan actions, and render selected evidence without automatically replacing assignments with unsupported parametric alternatives (Secs. 5.3–5.4; App. O.2).

Under matched evidence and rendering, Table 5 reports CRC 88.5 for RA-srFGW versus 85.8 for GPT-4o Joint Assignment. App. M.3 (Table 21) confirms +2.70 CRC [1.03, 4.44] across 200 paired stories.

This supports the additional value of explicit evidence-grounded structural correspondence over direct LLM joint assignment.

We appreciate the reviewer's suggestions, which have helped us strengthen both the controlled evaluation and the discussion of cultural-memory coverage.

Comment 2/3 — Q1–Q2: Fixed-reference ablation and full-benchmark human agreement

Q1. Does Flat Visual Memory use the same final images as Full?

No. Table 4(A) uses the same underlying visual archive for Flat Visual Memory and Full, but each method independently selects its final references (App. L.4).

To isolate structured conditioning from visual evidence, App. M.2 (Tables 19–20) evaluates a matched-input control on 50 stories × two cultures (100 conditions). All three variants receive the same six Full-selected images in identical order. Each independently generates its canonical frame with GPT-4o, followed by the same PixVerse V6 video backend under matched settings (Auxiliary Table M2).

Method (GPT-4o → PixVerse V6)CRV ↑CRC ↑
PixVerse V6 + Retrieved Images (Images-only)84.1079.70
PixVerse V6 + Independent Matching85.3082.80
CultureWeaver (PixVerse V6)86.1088.20

Images-only uses images and source narrative/actions without explicit cultural bindings. Independent adds node-wise cultural bindings without joint transport or target-relation instructions. Full adds RA-srFGW joint bindings and directed relational conditioning.

Table 20 reports Full–Independent +5.40 CRC [0.79, 10.24] and Full–Images-only +8.50 CRC [4.63, 12.42] (95% paired CIs). Both intervals exclude zero, whereas the corresponding CRV gains (+0.80/+2.00) are not statistically resolved.

Conclusion: identical references and generation backbones do not fully explain the relational gains; explicit cultural bindings and directed relational conditioning provide additional value. Table 4(B) separately evaluates matching strategies within the shared VCM pipeline (CRC 77.3 Independent vs. 88.5 Full).

Q2. Does human–automatic ranking agreement hold per scene, not just over seven aggregate method means?

Yes. We now evaluate all 400 scene–culture conditions, rather than a sampled subset. The blind full-video study includes 303 participants and 2,424 participant-condition evaluations (Table 2; App. H). Each condition receives six or seven independent rankings of all seven methods.

App. N.1 (Tables 24–25) reports within-scene human–automatic Spearman correlations:

Target cultureNPCCFCRVCRC
Chinese.677.704.759.826
Persianate.735.714.806.695
Macro-average.706.709.783.761

All four mean correlations have positive story-bootstrap 95% CIs (Table 25). Human inter-rater Kendall's W is .677/.726/.707/.664 for NP/CCF/CRV/CRC, respectively, supporting agreement among independent raters.

These full-benchmark results complement the method-level Spearman correlations in Table 3 (NP .839, CCF .946, CRV .821, CRC .888).

Separately, App. N.2 (Table 26) reports pre-aggregation Gemini–Claude exact agreement of 76.3%/73.7%/71.6%/73.0% for NP/CCF/CRV/CRC, with weighted κ=.605–.635.

Conclusion: human–automatic ranking agreement persists within individual generation conditions across the complete benchmark, rather than only among aggregate method means. Human inter-rater concordance and independent VLM agreement provide complementary reliability evidence.

Comment 3/3 — Q3–Q4: VCM construction reproducibility, ethics, and typo

Q3. Where are the prompts used to construct VCM?

App. L.2 provides the full construction-stage annotation instruction, input template, structured output specification, and human-review protocol, complementing Secs. 4.1–4.2 and Apps. I/J. Each request supplies target culture, source-unit ID, source image(s), original caption, metadata/provenance, and local context. The instruction extracts typed character/object/scene records, narrative functions, affordances, visual/contextual attributes, and directed subject–predicate–object relations with supporting image IDs and evidence explanations. Action, causal, and temporal information is retained as auxiliary event context, not additional transport nodes.

How is evidence quality controlled? The construction instruction distinguishes OBSERVED (direct source support) from INFERRED (uncertain model interpretation); VERIFIED is assigned only after human review. All retained model-derived annotations are reviewed against source evidence. Unsupported historical claims, invented cross-source relations, and story-specific hand-written replacements are excluded (App. L.2; Sec. 4.2). App. L.3 (Table 18) documents reproducible curation effort:

MemoryUnits screenedUnits retainedPerson-hours
Chinese36127692
Persianate19813066
Total559406158

Conclusion: VCM is a documented, human-reviewed, reusable cultural evidence resource, not an unreported prompt-generated graph or a set of manually specified Western-to-target replacements.

Q4. What are the concrete ethical risks and safeguards?

The expanded Ethics Statement and App. O.3 specify archival and elite-class bias, stereotypes, cultural/period overgeneralization, sensitive religious or historical representations, and source-image rights. Safeguards include retaining source provenance and OBSERVED/INFERRED/VERIFIED status, human review of retained cultural annotations, period-aware generation instructions, and respect for image licenses and attribution. We recommend explicitly labeling generated outputs as fictional adaptations and culturally knowledgeable review for sensitive or public-facing uses. Sec. 7 further defines the task boundary: this is evidence-grounded fictional narrative re-instantiation, not authoritative historical reconstruction. Conclusion: the proposed grounding makes cultural choices traceable and their representational scope explicit, while preserving the distinction between adaptation and historical authenticity.

Typo. The spelling at the cited location has been corrected to “operation” in Sec. 3. Thank you for these suggestions.

Reviewer_j43kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Narrative Cultural Re-instantiation: re-rendering a source scene as a video in a target culture, replacing characters and objects while preserving narrative functions and directed relations. CultureWeaver builds a Visual Cultural Memory (VCM) and abstracts the scene into a narrative graph. They select cultural correspondences via relation-aware semi-relaxed FGW, and render them with frozen image and video generators. They also propose a benchmark. On this benchmark, they outperform commercial video models and stylization baselines on four VLM-judged metrics and in a human study.

Critical strengths
  1. A clear and useful new task. The paper separates cultural re-instantiation, where narrative entities are replaced with functionally equivalent target-culture counterparts, from plain visual stylization. Prior work and current video generators mostly mix the two up.

  2. A benchmark and evaluation method for the task. The authors build the Western Fairy-Tale Short-Scene Benchmark. They also propose evaluation metrics, rather than only style similarity. The benchmark and evaluation protocol could be reused by future work on this task.

Critical weaknesses
  1. Potential limitation and failure cases are not discussed. The framework assumes every narrative element can be abstracted into a culture-neutral function and then realized in the target culture. This may fail in several cases: (1) The element has no meaningful counterpart in the target culture. (2) The element's narrative function depends on its source-culture meaning. (3) The retrieved elements come from different historical periods. The Chinese VCM, for example, runs from BCE to the Qing dynasty, so one scene could combine artifacts from incompatible eras. The paper does not say whether temporal or stylistic coherence across bindings is enforced, or how these failure cases are handled.

  2. Generalization. All experiments use historical Chinese and historical Persianate settings. Building each VCM appears to take substantial curation. It is unclear how well the method generalizes to other cultures, especially ones with sparse or less structured visual archives, or how much effort a new culture would cost.

  3. Possibly unfair comparison. Baselines get two fixed, culture-level images. CultureWeaver gets up to six scene-specific references chosen by its own retrieval and transport stages, plus an extra GPT-4o stage that generates a canonical frame before image-to-video generation. Part of the gain may come from these richer inputs. A baseline with the same number of retrieved references, or the same frame-then-animate pipeline, would make the comparison more convincing.

  4. A debatable example in Figure 3. The figure marks "courtly figure in a ruined hut" as a failure. In the source story the queen is disguised, though, so a high-status figure appearing in a humble dwelling is narratively plausible. This makes the example less convincing and raises a broader concern: the evaluation criteria may penalize realizations that are valid but unexpected.

  5. Limited technical novelty. The task is new and the results are strong. The method itself, though, combines known pieces: LLM-based graph abstraction, retrieval, fused Gromov-Wasserstein-style optimal transport, and frozen multi-reference image/video generators. The individual pieces are not particularly surprising.

  6. Minor. Typo at L135: "openration" should be "operation".

Questions for authors
  1. Action instructions and binding. How are the narrative action instructions π_N built from the preserved relations and event context? In the frozen realization stage (Sec. 5.4), how is the binding A* actually given to the image generator?

  2. Terminology. Do "flat image-text memory" (L424) and "Flat Visual Memory" (Table 4) mean the same ablation setting? If so, please use one term. If not, please explain the difference.

  3. Missing counterparts. How does the method behave when a source element has no suitable counterpart in the target VCM? Is there a fallback, such as keeping the element, abstracting it, or dropping it?

  4. Extension to video translation. The current method is text-to-video, conditioned on a source scene description. Could it be extended to video-to-video cultural translation, where an existing source video is re-instantiated in the target culture while keeping its motion and composition?

7 comments

Comment 1/3 — W1–W2: Failure boundaries, historical coherence, and generalization

Thank you for highlighting these scope and generalization questions. We address the three W1 cases separately.

W1(1). No meaningful cultural counterpart.

CultureWeaver does not require literal one-to-one equivalence between cultures. Sec. 3 defines cultural correspondence through narrative roles, functions, affordances, and directed relations. When an exact counterpart is unavailable, RA-srFGW preserves the source narrative requirements and identifies the best-supported functional realization among retrieved VCM records, jointly considering role suitability and relational coherence (Sec. 4.3; App. O.2). App. O.1 (Table 27) directly evaluates reduced functional coverage across 50 stories × two cultures: removing essential counterpart families decreases CRV by 2.85 and CRC by 5.34 points. These results demonstrate that cultural adaptation depends on available functional evidence, while the proposed correspondence mechanism supports flexible, function-based substitution rather than rigid cultural equivalence.

W1(2). Source-culture-specific narrative meaning.

CultureWeaver preserves narrative meaning through social roles, functions, required affordances, directed interactions, and causal/temporal event context (Secs. 3, 5.1–5.4). For example, the poisoned-apple scene is represented through deceptive offering, acceptance, consumption, and harm, rather than the apple's literal visual identity. This allows different cultural realizations to preserve the same narrative requirements. Table 4(B) supports the importance of explicit relational meaning: removing relation semantics reduces CRC from 88.5 to 78.4. Thus, the framework preserves the functional and relational meaning expressed in the narrative representation; source-specific symbolic meanings not captured by that representation require richer contextual annotation. This distinction is also clarified in the main-text Limitations paragraph (Sec. 7).

W1(3). Historical and stylistic coherence.

CultureWeaver incorporates source provenance, contextual attributes, visual evidence, and scene/period cues to guide cultural selection and frozen prompt planning (Sec. 4; App. J; Sec. 5.4; App. L.4). The planner can identify potential period or visual inconsistencies and refine rendering instructions. This provides period-aware coherence guidance rather than a global hard constraint across selected bindings. Importantly, the Chinese VCM's temporal range refers to historical/literary settings depicted, whereas Persianate dates mainly describe artwork production periods (Sec. 4.1; App. L.3). Our contribution is an evidence-grounded framework for fictional narrative cultural re-instantiation, rather than exact historical reconstruction; the corresponding scope and representational considerations are discussed in Sec. 7 and App. O.3.

Evidence-coverage analysis. App. O.1 (Table 27) further distinguishes functional evidence from relational evidence: retaining 50% of memory nodes decreases CRV/CRC by 2.90/4.44, while retaining 50% of relations decreases CRC by 4.06. These findings show that functional candidate coverage affects the availability of suitable cultural realizations, whereas relational coverage affects the coherence of their joint interactions. The original Table 4(A) establishes the contribution of external cultural grounding (CRC 88.5 Full vs. 79.2 Parametric-only), while Table 4(B) demonstrates the additional value of joint structural correspondence (88.5 Full vs. 77.3 Independent). Together, these experiments support the complementary contributions of evidence-grounded cultural memory and relation-aware matching.

We thank the reviewer for motivating a clearer statement of scope. Sec. 7 now includes a dedicated Limitations paragraph discussing omitted cultural meanings, unavailable counterparts, and the absence of hard cross-period constraints, with supporting analyses in Apps. O.1–O.3.

W2. Generalization and curation cost.

The Chinese paired narrative panels and Persianate individual artworks use the same memory schema, enrichment/human-review protocol, retrieval, RA-srFGW formulation, and decoding, without culture-specific parameter updates or handwritten Western-to-target mappings (App. J.6). This demonstrates procedural portability across two distinct archival formats; extension to other cultures depends on their available evidence. App. L.2 supplies construction instructions, while App. L.3 (Table 18) reports 559 source units screened, 406 retained, and 158 person-hours (92 Chinese; 66 Persianate). Curation is a reusable up-front investment, not a per-video intervention; Table 27 measures sensitivity to sparser memory support.

Comment 2/3 — W3–W6: Matched-input comparison, Figure 3, and technical contribution

W3. Are richer references and the extra first-frame stage responsible for the gains?

They do not fully explain the relational gains. App. M.2 (Tables 19–20) evaluates the matched-input control on 50 stories × two cultures (100 conditions). All variants receive the same six Full-selected images in identical order, with matched source conditions and rendering settings. GPT-4o separately generates each variant's canonical frame, followed by the same PixVerse V6 video backend (Auxiliary Table M2).

Method (GPT-4o → PixVerse V6)CRVCRC
PixVerse V6 + Retrieved Images (Images-only)84.1079.70
PixVerse V6 + Independent Matching85.3082.80
CultureWeaver (PixVerse V6)86.1088.20

Images-only uses the references and source narrative/actions without explicit cultural bindings. Independent adds node-wise bindings without joint transport or target-relation instructions. Full adds RA-srFGW joint correspondence and directed relational conditioning.

Table 20 reports Full–Independent +5.40 CRC [0.79, 10.24] and Full–Images-only +8.50 CRC [4.63, 12.42] (95% paired CIs). Both intervals exclude zero; the corresponding CRV gains (+0.80/+2.00) are not statistically resolved. Conclusion: explicit bindings and directed relational conditioning improve coherence beyond identical visual references.

Table 1 compares complete systems under their documented interfaces (App. F). Commercial baselines receive two fixed culture-level references from the generation-time VCM (App. L.1), whereas CultureWeaver selects up to six scene-specific references. This evidence selection is itself a contribution.

Table 4(B) isolates matching strategies within the shared VCM/rendering pipeline (CRC 77.3 Independent vs. 88.5 Full). Table 5 compares GPT-4o Joint Assignment with RA-srFGW under matched graphs, evidence, and rendering (CRC 85.8 vs. 88.5). App. M.3 (Table 21) confirms paired gains over GPT-4o Joint across 200 stories: +1.45 CRV [0.04, 2.92] and +2.70 CRC [1.03, 4.44].

Conclusion: structure-guided cultural evidence selection and coherent joint correspondence make complementary contributions.

W4. Is the "courtly figure in ruined hut" a valid disguise?

A high-status character in a humble dwelling is not, by itself, a failure. Sec. 3 explicitly permits narratively intended deception or transgression; the evaluation specifications in App. G assess required roles and directed interactions rather than banning unusual settings. Figure 3's negative example is a joint failure: the offerer and recipient are reversed, the direction of offering is incorrect, and overtly courtly/luxurious depiction does not convey the intended disguise and modest environment. The short label refers to this combined role, action, and context mismatch, not a general rule about who may enter a hut. The reversed directed interaction is decisive, illustrating why individual plausibility is insufficient.

W5. What is technically new beyond established components?

Our contribution is a task-specific, asymmetric, evidence-grounded correspondence formulation linking narrative requirements directly to cultural record and visual-evidence selection. RA-srFGW jointly optimizes functional compatibility, directed relation semantics/evidence confidence, and graph topology, preserving source mass while leaving candidate mass free. Identity-constrained decoding produces concrete cultural bindings (Secs. 4–5; App. I).

Table 4(B) validates these design choices: CRC 88.5 Full vs. 78.4 without relation semantics, 86.7 without topology, and 84.8 without semi-relaxation.

App. M.4 (Table 22) demonstrates the optimization trade-off: RA-srFGW achieves near-Beam diagnostic cost (0.2940 vs. 0.2913) with substantially lower median runtime (0.394 vs. 1.418 s; ~3.6× faster).

The complete 200-story Full–Beam comparison (Table 21) shows no statistically resolved downstream CRV/CRC difference (−0.75 [−1.86, 0.37] / +0.60 [−0.91, 2.16]).

Conclusion: joint relational optimization improves cultural correspondence, while RA-srFGW provides an efficient semi-relaxed transport formulation, rather than universal superiority over other joint optimizers.

W6. Typo. "openration" has been corrected to "operation" in Sec. 3.

Comment 3/3 — Q1–Q4: Realization interface, flat-memory terminology, missing counterparts, and video-to-video extension

Q1. How are action instructions built, and how is the binding supplied to generation?

Secs. 5.1–5.4 and App. L.4 specify the interface. The narrative graph preserves typed character/object/scene roles and directed relations; auxiliary event context preserves actions, temporal order, and causal dependencies. Frozen prompt planning compiles these into ordered action instructions pi_N, specifying who acts, who receives the action, its direction, and its sequence. RA-srFGW supplies the discrete binding A* (narrative role → selected cultural record); the selected records supply role/function/affordance descriptions and up to six supporting images R*. GPT-4o uses the binding, references, and instructions to generate the canonical frame; PixVerse V6 uses that frame and the action instructions for the video. In Figure 4, Returner→Scholar, Recipient→Master, Returned Object→Jade Pendant, Setting→Mansion Hall, while the instructions preserve Scholar returns pendant to Master; Master then invites Scholar. Conclusion: both target-cultural identities and narrative action direction are explicitly transmitted to frozen generators, rather than left to unconstrained prompting. App. M.1 (Auxiliary Table M1) audits this interface: 15 changes among 700 role/relation units (2.14%) across 100 conditions.

Q2. Are "flat image–text memory" and "Flat Visual Memory" the same?

Yes. Both refer to the Flat Visual Memory ablation in Table 4(A); App. L.4 clarifies its implementation. It uses the same underlying image archive as Full, with image/text metadata and functional descriptions but without the VCM's directed relation graph or pairwise topology optimization. Its node-wise retrieval/matching can select different final images from Full. Conclusion: Table 4(A) tests the overall value of structured memory/evidence selection (CRC 80.4 Flat vs. 88.5 Full); Table 4(B) separately tests structural matching. We use "Flat Visual Memory" as the canonical baseline name.

Q3. What happens when no exact cultural counterpart is available?

CultureWeaver preserves the source narrative's required role, function, and directed interactions, and searches the retrieved VCM for the best-supported functional realization, rather than requiring a literal cultural equivalent (Sec. 4.3; App. O.2). The RA-srFGW objective jointly considers functional compatibility and relational coherence, allowing culturally different entities to fulfill the same narrative role. Missing relational evidence receives a finite penalty of 0.90, so candidates from different cultural source units can still be combined (App. I.1).

App. O.1 (Table 27) further evaluates this mechanism under reduced cultural-memory coverage. Removing essential counterpart families decreases CRV by 2.85 and CRC by 5.34 points, while reducing memory nodes or relations produces distinct effects on functional validity and relational coherence.

These results show that CultureWeaver supports flexible, evidence-grounded cultural substitution rather than rigid one-to-one replacement, while identifying how the availability of functional and relational evidence influences adaptation quality.

Q4. Could this extend to video-to-video cultural translation?

Yes, as a natural extension of the narrative-level matching interface. A video-conditioned front end could extract tracked characters/objects, directed interactions, actions and temporal order from the source video, then apply the same VCM retrieval and RA-srFGW correspondence. A motion-conditioned renderer could use source-video trajectories, pose, camera motion, timing, and composition while changing the culturally realized characters, objects, and settings. For example, an offerer could change cultural identity without reversing who offers what to whom. This is a proposed extension, not an evaluated capability of the current text-conditioned pipeline; temporal tracking and motion-preserving rendering would be additional components. The key reusable contribution is correspondence over narrative functions and directed relations, independent of the source medium.

Reviewer_2BqvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes to re-render short fairy-tale scenes as videos set in a different culture (historical Chinese or Persianate), substituting characters, objects and settings while preserving the plot. The authors argue that these substitutions must be chosen jointly rather than independently. The pipeline is training-free: it retrieves candidates from a curated, annotated image memory, selects a consistent set via relation-aware semi-relaxed FGW transport, and renders the output with GPT-4o plus a commercial video model. On a 200-scene benchmark, using VLM-judged metrics and a human study, it outperforms commercial video generators, a culture-aware prompting method, and style-conditioned baselines on the narrative and cultural metrics.

Critical strengths
  1. The task framing is the main contribution. Much of the work on cultural generation mostly checks whether an output looks like Chinese or Persian. This paper asks a different question: does the adapted scene still do what the original scene did? For example, the poisoned object can still be offered and eaten, and the deceiver can still plausibly approach the victim. I find this reframing the most valuable part of the paper.

  2. The paper makes joint coherence measurable. Figure 3 gives a concrete example showing that substitutions which are fine one by one can clash when combined. Because the benchmark annotations are culture-neutral, this clash becomes something you can actually measure, and I think other people working on cultural adaptation will find this framing useful.

  3. The documentation is complete. The appendix covers source provenance, every baseline prompt, every judge prompt, and hyperparameter sensitivity. This makes the work easier to check and to build on.

Critical weaknesses
  1. The comparison with baselines doesn't isolate the main idea. According to App. F, the commercial baselines get the same two fixed style images per culture for every scene. CultureWeaver gets up to six reference images picked for each specific scene from its own memory (App. I), and also goes through an extra GPT-4o step to make the first frame. That gives a lot more help. Table 4 suggests this is where most of the gain comes from: the parametric-only version already gets CRC 79.2, about the same as the commercial baselines (~80), and the big jump to 88.5 only shows up once the curated memory is added. The structural matching (main claim of the paper), beats a GPT-4o joint assignment by only 1.5–2.7 points (Table 5), and there's no significance test or variance for that comparison. To really know how much the "joint structural correspondence" matters, I'd want to see a baseline that gives the same retrieved images to the same renderer but skips the transport step.

  2. The evaluation relies mostly on the authors' own VLM-judged metrics. All four main metrics, say NP, CCF, CRV, CRC, were designed by the authors and are scored by two VLMs that look at five frames and give 0, 0.5 or 1. CCF, CRV and CRC also compare against a set of museum images, which may favor a method that is itself conditioned on similar historical imagery. There's no human check of how accurate the VLM judges are, and no report of how often the two judges agree. The human study helps, but it doesn't report agreement between raters, and the correlations in Table 3 are based on ranking just seven methods. Together with point 1, I'm not yet convinced the gains come from better cultural adaptation rather than from better-matched inputs and evaluation.

Questions for authors

Questions:

  1. Same references, no transport. What happens if a commercial model (e.g., Seedance 2.0 with multi-image input) gets the same up-to-six reference images that your retrieval picks? Or what if you run "retrieval + GPT-4o first frame + video model" but skip the RA-srFGW step? This is the comparison I most want to see, since right now the baselines only get two fixed style images per culture.

  2. Is the transport step really needed? The narrative graphs are small (344 relations over 200 scenes, about 1.7 per scene), with 12 candidates per node. Did you try just searching all combinations under the same cost, or a simple rule like "pick the one source image that covers the most roles"? Also, in the Chinese memory, 1,840 of the 2,412 edges (~76%) are context-derived from the same source unit. How much of the structural term ends up meaning "pick candidates from the same picture"?

  3. How much can we trust the VLM judges? Could you report a small human check of the judges' scores? I'd also like to see how often Gemini and Claude agree with each other. And does the method ranking stay the same without the museum reference images?

Clarification:

  1. Where do the two style reference images given to the baselines come from? Are they from the same sources as your cultural memory? This affects how to read the CSD numbers.

  2. For the human study, what was the agreement between raters?

8 comments

Comment 1/3 — Critical Weaknesses 1–2: Structural Contribution and Evaluation Validity

W1. Does joint correspondence contribute beyond richer reference images?

Yes, particularly to relational coherence. App. M.2 (Tables 19–20) tests the requested matched-input control on 50 stories × two cultures (100 conditions). All variants use the same six Full-selected images in the same order; each independently generates its first frame with GPT-4o and video with PixVerse V6, under matched settings (Auxiliary Table M2).

Method (GPT-4o → PixVerse V6)CRVCRC
Retrieved Images (Images-only)84.1079.70
Independent Matching85.3082.80
CultureWeaver (Full)86.1088.20

Images-only provides images and source narrative/actions without explicit cultural bindings; Independent adds node-wise bindings but no joint transport or target-relation instructions; Full adds RA-srFGW joint bindings and directed relational conditioning. Table 20 reports +5.40 CRC [0.79, 10.24] over Independent and +8.50 CRC [4.63, 12.42] over Images-only (paired 95% CIs). Both CRC intervals exclude zero; the corresponding CRV gains (+0.80/+2.00) are not statistically resolved. Thus, identical images and generators do not fully account for the gains in relational coherence.

Does the matching mechanism itself matter? Table 4(B) holds the VCM/rendering pipeline fixed: Full improves CRC 77.3→88.5 over Independent, while removing relation semantics reduces CRC to 78.4. Table 5 compares RA-srFGW against GPT-4o Joint Assignment under matched graphs, evidence, and rendering (CRC 88.5 vs. 85.8). App. M.3 (Table 21) confirms paired gains over GPT-4o Joint on 200 stories: +1.45 CRV [0.04, 2.92] and +2.70 CRC [1.03, 4.44]. These comparisons support structured correspondence beyond reference selection.

How should Table 1 be interpreted? It compares complete systems using their documented native interfaces (App. F). Commercial image-capable baselines receive two fixed culture-level style references from the generation-time VCM (App. L.1), while CultureWeaver selects up to six scene-specific references. This evidence selection is part of the method. Table 4(A) separates the memory/representation contribution (CRC 79.2 Parametric-only, 80.4 Flat, 88.5 Full), while the matched-image control isolates additional relational conditioning.

W2. Are the automatic metrics supported by independent human evaluation?

Yes, at the full-video ranking level. Table 2 and App. H report 303 blind human participants, 2,424 participant-condition evaluations, and all 400 scene–culture conditions, each with six or seven complete rankings. App. N.1 (Tables 24–25) now measures human agreement over the entire 400-condition benchmark:

MetricHuman Kendall's W [95% CI]Human–automatic Spearman ρ [95% CI]
NP.677 [.639, .709].706 [.687, .725]
CCF.726 [.688, .764].709 [.690, .728]
CRV.707 [.681, .733].783 [.766, .799]
CRC.664 [.625, .713].761 [.745, .776]

These are within-condition agreement statistics, not correlations inferred solely from the seven aggregate method means in Table 3. They show consistent human rankings and positive scene-level alignment with the automated metrics.

Do Gemini and Claude agree? App. N.2 (Table 26) analyzes their original pre-aggregation atomic judgments. Exact agreement is 76.3% NP, 73.7% CCF, 71.6% CRV, and 73.0% CRC, with linear weighted κ=.605–.635. This measures inter-VLM reliability; the complete-video human study independently validates ranking alignment, not the correctness of every individual atomic judgment.

Could museum references favor CultureWeaver? App. G.7 uses a held-out, image-deduplicated museum corpus and identical evaluation evidence for all methods within each scoring unit. App. N.3 (Table 27) re-scores 40 conditions without museum images/metadata: Full's CRV changes by −0.88 [−2.50, 0.88] and remains ranked first; CCF changes by −2.62 [−4.38, −0.88], with its rank shifting from first to tied second. Thus, the CRV ranking is retained in this check, while cultural-content scoring shows measurable evidence sensitivity.

We appreciate these suggestions, which have helped strengthen the controls and evaluation transparency.

Comment 2/3 — Q1–Q2: Same-reference control, inference graphs, and optimizer alternatives

Thank you for these specific suggestions. We address the matched-input comparison and alternative optimizers with the experiments in the revised Appendix M.

Q1. What happens with the same retrieved images but no RA-srFGW?

App. M.2 (Tables 19–20) provides the requested control on 50 stories × two cultures (100 conditions). For each condition, all variants receive the same six Full-selected images in identical order. Each uses GPT-4o to generate its own canonical frame, followed by PixVerse V6 under matched rendering settings (Auxiliary Table M2).

MethodCRVCRC
Retrieved Images (Images-only)84.1079.70
Independent Matching85.3082.80
CultureWeaver (Full)86.1088.20

Images-only uses references and narrative/actions without explicit cultural bindings. Independent adds node-wise bindings but no joint transport or target-relation instructions. Full adds RA-srFGW joint bindings and directed relational conditioning.

Table 20 reports Full–Independent +5.40 CRC [0.79, 10.24] and Full–Images-only +8.50 CRC [4.63, 12.42] (95% paired CIs). Both CRC intervals exclude zero; the corresponding CRV gains (+0.80/+2.00).

Conclusion: structured role and relation conditioning improves relational coherence after reference selection, even with identical images and generators. Table 4(B) separately isolates matching mechanisms in the shared VCM/rendering pipeline.

Q2. Are inference graphs only 344 relations across 200 scenes?

No. The 344 relations are held-out CRC evaluation annotations (App. E.6), not inference-time graphs. In 100 audited scene–culture conditions, inference graphs average 6.44 narrative nodes and 7.44 directed edges, with 44.32 distinct cultural candidates (Sec. 5.1; App. M.4, Auxiliary Table M3). K=12 is the retrieval budget per narrative node. Joint assignments must also respect relation directions and distinct character identities. Consistently, Table 4(B) reports CRC 77.3 Independent vs. 88.5 Full, falling to 78.4 without relation semantics.

Were alternatives to transport, including exhaustive search, tested?

Yes. App. M.4 (Table 22) compares Independent, discrete Beam, RA-srFGW+decode, and Exact Search using the same discrete diagnostic objective and candidate/identity constraints (distinct from the transport objective). Exact Search certifies optimality in 40/100 instances within 60 s; on these common 40 cases:

OptimizerMean diagnostic cost ↓Median time (s) ↓
Independent0.38800.054
Beam0.29131.418
RA-srFGW + decode0.29400.394
Exact (certified)0.24427.450

RA-srFGW achieves near-Beam cost at ~3.6× lower median solve-and-decode time on this subset. Importantly, Beam is also a relation-aware joint optimizer.

The complete 200-story downstream comparison (Table 21) reports Full–Beam −0.75 CRV [−1.86, 0.37] and +0.60 CRC [−0.91, 2.16].

Conclusion: joint structural optimization matters, while RA-srFGW's demonstrated advantage over Beam is matching-stage efficiency—not universal downstream quality superiority.

Does the structural term simply prefer one source image?

No same-image rule is imposed. Although 1,840/2,412 Chinese VCM edges are context-derived (App. J.4), a missing target relation receives a finite 0.90 cost, and Full bindings span 3.04 original source units on average (Apps. I.1, M.5; Auxiliary Table M6).

Holding candidates and evidence-linked edges fixed, removing context-derived relations decreases CRC by 2.93 [1.27, 4.55] (Table 23, 95% CI).

Thus, context-derived directed relations contribute to coherence beyond simply grouping candidates from one picture.

Comment 3/3 — Q3 and Clarifications 4–5: VLM Reliability and Style-Reference Provenance

Thank you for your valuable suggestions. We have expanded the evaluation reliability analysis to the complete human study and further examined VLM agreement and museum-evidence sensitivity.

Q3. How reliable are the two VLM judges?

App. N.2 (Table 26) evaluates Gemini 2.5 Pro and Claude Sonnet 5 using their original pre-aggregation atomic judgments across the benchmark. The results are:

MetricPaired decisionsExact agreementLinear κ
NP9,04476.3%.613
CCF11,21473.7%.635
CRV11,21471.6%.605
CRC4,81673.0%.620

These results establish inter-VLM consistency, rather than directly measuring atomic accuracy against human-adjudicated labels. We separately validate the resulting automatic metrics against independent human judgments of complete videos.

Does human–automatic agreement hold at the individual-scene level?

Yes. App. N.1 (Tables 24–25) now evaluates all 400 scene–culture conditions, covering 303 participants and 2,424 human evaluations. Mean within-scene Spearman correlations are NP .706, CCF .709, CRV .783, and CRC .761, with positive story-bootstrap 95% CIs for all four metrics.

Conclusion: the automated metrics show positive alignment with independent human rankings across the complete benchmark, beyond the aggregate seven-method comparison in Table 3.

Does removing museum references preserve method rankings?

Partially. App. G.7 uses a held-out, image-deduplicated museum corpus, providing identical evaluation evidence to all methods for the same evaluation unit.

App. N.3 (Table 27) re-scores 40 conditions without museum images or metadata. Full changes by CCF −2.62 [−4.38, −0.88], CRV −0.88 [−2.50, 0.88], and CRC −2.12 [−3.62, −0.50]. Full remains first in CRV, while its CCF rank changes from first to tied second.

Conclusion: functional ranking remains stable in this control, while cultural-content scores show measurable sensitivity to external museum evidence.

C4. Where do the two baseline style images come from?

App. L.1 confirms that both are fixed culture-level references selected from the same generation-time VCM used by CultureWeaver. The pair is reused across all stories in each target culture and supplied to commercial two-image baselines. Single-image style baselines receive the designated first anchor, while MAVEN remains text-only.

The same reference pair defines the CSD target for every method, ensuring a common style-evaluation reference. These anchors are separate from the held-out museum evaluation corpus.

C5. How consistent are human raters?

App. N.1 (Tables 24–25) now reports inter-rater agreement from the entire human study, rather than a sampled subset. All 400 conditions have six or seven independent human rankings.

Mean within-condition Kendall's W is NP .677 [.639, .709], CCF .726 [.688, .764], CRV .707 [.681, .733], and CRC .664 [.625, .713] (95% story-bootstrap CIs).

These results demonstrate positive agreement among independent human raters across the complete benchmark, strengthening the reliability of the human evaluation and its validation of automated metric rankings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.