acceptodds
Under review as a conference paper at ICLR 2027

VANISH: Fusing Source Code and Virtual Assembly for Robust Cross-Language Code Clone Detection

Abstract

Cross-language code clone detection supports code search, migration, and plagiarism analysis, yet remains vulnerable to semantics-preserving obfuscation because existing models rely heavily on lexical evidence. We propose VANISH, a dual-view framework that combines multilingual source code with identifier-reduced virtual assembly. VANISH resolves supervision, tokenization, and granularity mismatches, trains both views independently with cross-language contrastive learning, and fuses their similarity scores. We also construct XL-Asm, a leakage-controlled benchmark of 91,863 programs from 29,324 functionality groups across 12 languages. VANISH achieves state-of-the-art performance with 0.674 X-MRR, outperforming CodeSage and UniXcoder by 12.3% and 68.5%, respectively. Under strong obfuscation, it recovers detection AUC from 0.85–0.88 to 0.94–0.95, gaining up to 8.9 percentage points (*p* < 0.001) without reducing clean-code performance. Results across languages, source encoders, real obfuscators, and adaptive attacks establish the *headroom principle*, whereby virtual assembly contributes most when source representations are degraded.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Privacy Applications, Code Clone Detection, Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Kb5qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VANISH, a framework for cross-language code clone detection that combines features from the source code with features from an LLM-generated pseudo-assembly counterpart. With the complementary assembly view, VANISH aims to tackle the challenges caused by real-world code obfuscation. On XL-Asm, a new dataset crafted by the authors, VANISH shows higher detection accuracy and robustness compared with existing baseline models.

Critical strengths
  1. The paper targets a practical problem, cross-language code clone detection, which is important for code reuse and retrieval in software development.
  2. The authors find that the compiled representation of source code can serve as a complementary detection signal that improves performance under obfuscation, and propose a solution that combines source code and assembly code for detection.
  3. Experiments on a newly crafted multilingual dataset demonstrate the effectiveness of the proposed method compared with several existing baseline models.
Critical weaknesses
  1. It is unclear why the authors choose x86-64 pseudo-assembly as the only compiled representation in VANISH, rather than other representations such as LLVM IR, which existing studies [1-2] have already used for cross-language code matching. The authors should provide an ablation to justify this choice.
  2. The paper lacks justification for deliberately treating same-language clones as negatives during training.
  3. The experiments lack an LLM re-write attack. As more and more code is processed by LLM agents, the proposed method should be shown to be robust to LLM re-writes of the code in order to survive this real-world challenge.
  4. Potential comparison unfairness and selectively reported numbers. First, in Table 1 the baselines are used off the shelf while VANISH is fine-tuned on the proposed dataset, which makes the comparison unfair. Second, the AUC recovery reported in the abstract is measured on a subset rather than the full test set; the results on the full test set may be much lower.
  5. The writing and presentation of the paper could be improved. For example, Figure 1 should separate the dataset generation part and the VANISH training/deployment part into two figures for clarity.

[1] Gui, Yi, et al. "Cross-language binary-source code matching with intermediate representations." IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). 2022.

[2] TehraniJamsaz, Ali, Hanze Chen, and Ali Jannesari. "GraphBinMatch: Graph-based similarity learning for cross-language binary and source code matching." IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). 2024.

Questions for authors
  1. The full VANISH inference time is much longer than that of the source-code-only baselines. Did the authors try a smaller ViC model for faster assembly generation? Or is it possible to use existing compilation tools instead of the ViC model on the supported languages to reduce the inference cost?
7 comments
AuthorsOpenReview ↗

Response to Reviewers

Thank you for the review. All four experiments report null/negative findings. CIs: 95% paired task-cluster bootstrap (2,000 resamples), multiplicity-uncorrected.

W1: Pseudo-assembly vs. LLVM IR

A frozen source branch is fused with clang LLVM IR/native assembly or stored ViC. Auxiliary encoders share data/protocol; test: 254 C/C++ records/127 tasks. Source/auxiliary token limits: 256/512; truncation: 89.8–98.8%. Clean-dev weights favor source-only on ties. Scores: X-MRR pp.

ViewStandaloneFused Δ vs. source (68.10)α
LLVM IR11.170.001.0
Native assembly19.11−0.60 [−2.99, +1.79]0.8
ViC10.790.001.0

Native assembly leads standalone: +8.32 [+2.78, +13.66] over ViC; +7.94 [+2.17, +13.22] over IR. IR−ViC: +0.37 [−4.47, +5.05] (ns). No view improves fusion on this narrow, truncation-limited cohort.

We claim toolchain-free availability, not superiority. Unlike XLIR/GraphBinMatch’s shared IR, virtual compilation needs no per-language compiler pipeline. Ours covers C/C++, not the other ten languages; audited clean parsing: 68–86%. This limits implementation, not compiler existence. Stored ViC coverage reflects construction, not 100% fresh-generation success.

W2: Same-language negatives

Only the positive mask changed: cross-language vs. all same-task non-self positives. Training: 600 updates/arm, 2,098 groups; test: 15,711 records/3,728 tasks. X-MRR: 68.28/68.69, +0.41 pp [−0.17, +0.96]. Queries improved/harmed: 19.8%/17.4%.

Neither policy significantly leads. Cross-language positives are evaluation-aligned, not superior or universal in historical checkpoints. This single-seed, bounded-update test cannot establish converged equivalence.

W3: LLM rewriting

One fixed-prompt Python rewrite/program; originals/rewrites tested on ≤20 official IO cases. Of 80 tasks, 60 originals passed, 50 pairs verified: 37 queries/179 candidates. ViC accepted: 31/37 queries, 149/179 candidates.

Clean→rewritten changes use 24 queries/149 candidates with common accepted views; fusion gains use 37 queries/179 candidates, full-gallery source fallback if either required auxiliary view is rejected. Clean-dev weights stay fixed.

ResultΔ X-MRR pp [95% CI]
Source, clean→rewritten+1.52 [−14.98, +19.30]
Assembly, clean→rewritten−15.60 [−30.89, −1.83]
Normalized source, clean→rewritten−4.65 [−21.93, +14.06]
Source + assembly vs. source−4.27 [−12.75, +3.84]
Source + normalization vs. source+0.65 [−2.54, +4.21]

Full 37-query source X-MRR: 53.58→53.92 (+0.34 pp; no paired CI). Common-cohort assembly: 25.00→9.40; R@1: 20.83→0.00. Rewriting exposes assembly sensitivity without measured source decline. Renaming robustness ≠ structural-rewriting invariance. One model/prompt/temperature-zero sample cannot characterize adaptive attackers; sampled IO agreement cannot prove semantic equivalence.

W4: Fairness and AUC

Adaptation: Matched budgets replace unqualified SOTA and +12.3%/+68.5% claims: same data, 1,200 updates, batch 16, seed 719, dev selection. Adapted CodeBERT/C4: 51.76/45.25 X-MRR vs. adapted CodeSage source’s 73.11. C4 adapts the objective, not the full system. Capacity: CodeSage ≈356M vs. CodeBERT-base ≈125M.

Scope: Subset AUC recovery is no general headline. Full clean test (15,711 records; 31,422 pairs): AUC 0.9900→0.9945; X-MRR +0.36 pp [+0.06, +0.65]. This is not obfuscated-subset recovery or proof of full-obfuscated-test robustness. Fresh clean record-level gains lose significance after program deduplication (H.3). Clean C/C++ parsing: 62/80 and 57/80 vs. perturbed 13/80 and 0/80. Older attacks are historical diagnostics, not verified semantics-preserving robustness evidence.

Provenance: Historical source training/checkpoint selection included 530/144 final-test groups, respectively; assembly selection used test X-MRR (H.2). This corrects assigning 530 groups solely to checkpoint selection. Affected results are qualified; dev-only selection cannot erase exposure. Fresh controls use audited, overlap-filtered training data and dev-selected checkpoints.

W5: Presentation

Figs. 1–2 separate construction from training/deployment; H reports cohort sizes/generation-success counts.

Q1: Smaller ViC or compilers?

Smaller ViC is untested. Sequential uncached probe median latency: 27.81 s for 9 successes; 32.75 s for all 10 attempts vs. source-only retrieval’s 0.022 s. Median paired ratio: 1,473×. Clang assembly p50: 0.049/0.403 s for dev C/C++; 0.786 s for test C++. Lower cost/stronger standalone W1 results motivate compiler-first processing with virtual fallback where unsupported; this hybrid is untested. Neither real nor virtual views improve cohort fusion: cheaper generation ≠ retrieval benefit.

Revisions

We added representation/positive-mask ablations and IO-tested rewriting, clarified scope/provenance/costs, and split construction/deployment diagrams (H.2–H.7; §§4.1–4.2, 4.5–4.6; Figs. 1–2).

AuthorsOpenReview ↗

Gentle Follow-up

Dear Reviewer,

With the rebuttal period now more than halfway through, we wanted to check whether you have had an opportunity to review our responses and updated manuscript. The revised PDF includes our additional experiments and clarifications, with changes marked in blue.

We would welcome your feedback on whether these additions resolve the issues you raised, and we remain available to address any outstanding questions. If the new results and revisions have sufficiently addressed your concerns, we kindly ask you to consider increasing your score. We sincerely appreciate your time and careful consideration.

Thanks for the rebuttal from the authors. Here are some follow-up questions and suggestions:

W1: The truncation rates (89.8–98.8%) in the W1 experiments are quite high. Could the lack of fusion gains be due to information loss caused by truncation?

W3: Could the authors compare the proposed method and its variants with the other baselines reported in the paper on the same LLM-rewritten data? This would clarify whether VANISH still outperforms the other baselines despite its observed performance decline on rewritten data.

W4: Given the disclosed test-set exposure, the main results in Table 1 should be replaced with results obtained using models trained on test-disjoint data.

Reviewer_BBRKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose VANISH, which addresses cross-language code clone detection under source-level obfuscation by combining source code with function-level virtual assembly. They also present XL-Asm, a leakage-controlled benchmark of 91,863 programs constructed from 29,324 functionality groups and 12 programming languages. Performance gain over CodeSage and UniXcoder as well as detection AUC improvement under strong obfuscation are reported.

The paper proposes a headroom principle, stating that an auxiliary assembly view is most useful when the source branch still has substantial residual error. The empirical analysis is useful, although the relationship between this principle and existing ideas on complementary views and ensemble error correction should be clarified.

Critical strengths

The reported experiments show that adding virtual assembly improves robustness over the same source encoder under several tested obfuscation settings, supporting the practical value of combining the two representations.

The analysis of when fusion helps or hurts provides useful insight into its benefits and trade-offs. And the paired XL-Asm resource supports further research on source–assembly representations.

Critical weaknesses

Apparent inconsistencies in the datasets descriptions and evaluation records make the main comparisons difficult to sustain. In particular, it is not clear whether the fusion weight was selected independently of the test results.

It lacks in-depth comparisons with state-of-the-art cross-language clone-detection methods. The advantages and the novelty concerned with existing source–assembly approaches need to be further clarified.

Questions for authors

The authors propose VANISH, which addresses cross-language code clone detection under source-level obfuscation by combining source code with function-level virtual assembly. They also present XL-Asm, a leakage-controlled benchmark of 91,863 programs constructed from 29,324 functionality groups and 12 programming languages. Performance gain over CodeSage and UniXcoder as well as detection AUC improvement under strong obfuscation are reported. The paper proposes a headroom principle, stating that an auxiliary assembly view is most useful when the source branch still has substantial residual error. The empirical analysis is useful, although the relationship between this principle and existing ideas on complementary views and ensemble error correction should be clarified. However, the following issues still need further clarification and revision:

i) Section 3.2, Appendix C.2, and Table 9 report different dataset counts or scopes. For reproducibility, it is necessary to clarify the association between programs and training entries as well as where the one-entry and cap=2 rules apply.

ii) In Table 8, Table 28 is specified as a full-test weight sweep, while in Appendix D.7, it is indicated that the fusion weight a=0.6 was selected on the development set via the sweep in Table 28.

iii) The main comparisons primarily cover general-purpose source encoders rather than the dedicated X-CCD methods discussed in Section 2. Including representative dedicated methods, or alternatively explaining comparison limitations and qualifying the state-of-the-art claim, would strengthen the evaluation.

iv) The authors describe the adaptations of ViC and CLAP, but the boundaries between methodological novelty, component integration, and empirical contributions could be clearer. Relating the headroom principle more explicitly to prior work on ensemble complementarity would also improve the positioning.

2 comments
AuthorsOpenReview ↗

Response to Reviewers

Thank you for the suggestions. We clarify data/selection, add dedicated X-CCD controls, and refine our contribution. CIs: 95% paired task-cluster bootstrap (2,000 resamples), uncorrected for multiplicity.

W1 & Q(i): Dataset accounting

We distinguish inventory, training records, and evaluation pools:

QuantityCount
Corpus482,553 source–assembly records; 29,324 groups; 12 languages
Train/dev/test groups23,733 / 1,863 / 3,728
Training inventory364,895 records
Capped dev pool7,721 records; 1,863 groups
Canonical test15,711 records; 3,728 groups; 13,820 unique programs
Evaluation samplingcap=2 per problem–language pair; seed 42

Cap=2 limits evaluation records, not distinct programs or the corpus. A program may recur with different generated assembly.

We replaced blanket “one-entry” wording with stage-specific rules. Neither 91,863 nor approximately 414k matches stored manifests; both are withdrawn as corpus sizes, replaced by audited counts (H.1). Records ≠ unique programs; repeated draws are training events. Group-disjointness does not exclude residual source overlap. Fresh clean record-level gains lose significance after program deduplication (H.3).

W1 & Q(ii): Selection versus test sensitivity

Table 28 is a full-test sensitivity sweep, not a dev-selection record. D.7 labels its optima post hoc.

New controls fuse frozen embeddings with α ∈ {0, 0.1, …, 1}, selected by X-MRR only on full clean dev data; ties favor the base backbone. Weights stay fixed across test attacks. No independent dev-selection record establishes the original α=0.6.

Historical source training/checkpoint selection included 530/144 final-test groups, respectively; the assembly checkpoint was selected by test X-MRR (H.2). This corrects our earlier attribution of 530 groups to checkpoint selection alone. Affected results are qualified; dev-only weight selection cannot erase training/selection exposure.

Fresh controls use declared pretrained backbones, remove 442 source-overlap groups from audited training data, and select checkpoints on dev metrics. Training manifests have no final-dev/test group-label overlap; this neither validates historical results nor excludes cross-resource semantic duplication.

W2 & Q(iii): Dedicated X-CCD comparison

We added an adapted C4 objective, not a complete C4 reproduction, under matched budgets: same data, 1,200 updates, batch 16, seed 719, dev selection. Adapted CodeBERT/C4 reach 51.76/45.25 X-MRR versus 73.11 for adapted CodeSage source.

On 15,711 records/3,728 tasks:

Auxiliary view for adapted C4X-MRR gain (pp)
Virtual assembly+4.78
Normalized source+14.11
Second source seed+27.96

The second-source auxiliary is a separately trained CodeSage, not C4, seed; capacity also changes. This extends evaluation to dedicated X-CCD and alternative views. Claims concern tested configurations, not unqualified SOTA. Budgets are matched, capacity is not: CodeSage ≈356M versus CodeBERT-base ≈125M parameters. ZC3, AdaCCD, and CLCDSA remain untested.

Q(iv): Contribution and prior approaches

ViC, CLAP, supervised contrastive learning, and linear fusion are existing foundations. VANISH contributes task-specific integration: function-level virtual compilation, tokenizer-compatible normalization, cross-language supervision, independent branches, paired-data construction, and robustness analysis—not a new compiler or fusion operator. Relative to real compilation, the distinction is auxiliary-view availability when the toolchain cannot process an input: coverage, not representation superiority.

On 254 records/127 C/C++ tasks, LLVM IR and ViC receive zero auxiliary weight; native-assembly fusion changes X-MRR by −0.60 pp [−2.99, +1.79]. Under compile-verified renaming, fusion gains 4.51 pp [1.53, 8.42] over source-only (55 queries, full gallery, source fallback). All 110 transformed programs recompile with canonicalized LLVM IR and native assembly byte-identical to originals. Our robustness claim targets this verified setting.

Headroom specializes ensemble complementarity empirically. For H=1−MRR_source and Δ=MRR_fused−MRR_source, Δ≤H describes opportunity, not guaranteed gain; the shared baseline creates mathematical coupling.

Net gain is rescue−harm. E3/E4 rescue is similar (6.61/6.66 pp), but harm differs (9.14/5.76 pp). This requires labels, is not an inference-time predictor, and E4’s net-gain CI includes zero. Gains rise with headroom for all three views; second-source ensembling leads on clean CodeSage, while normalization gives comparable measured renaming gains at lower cost. Headroom describes second-view opportunity, not assembly-specific superiority.

Paper revisions

We audited counts/selection, added matched-budget CodeBERT/C4 comparisons, and qualified novelty, SOTA, and headroom claims (Sections 3.2, 4.1–4.2;Appendix H.1–.4).

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer,

We sincerely appreciate the time and effort you have devoted to reviewing our paper and considering our responses.

With the discussion deadline approaching, we would like to leave the door open for any further questions or concerns. If there are aspects of our method, additional experiments, or revisions that you would like to explore in more depth, please feel free to let us know. We would be glad to continue the discussion and provide any further clarification.

Thank you again for your thoughtful feedback and valuable contribution to improving our work.

Reviewer_1fjSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the robustness of cross-language code clone detection under semantics-preserving code obfuscation. The authors propose VANISH, a framework that uses ViC to convert source code into virtual assembly. The source-code and assembly encoders are trained independently using supervised contrastive learning. Their similarity scores are then combined through weighted fusion for retrieval. The paper also introduces XL-Asm, a paired source-code and virtual-assembly dataset. Finally, it proposes the “headroom principle,” which suggests that the assembly view offers greater benefits when source-code representations leave more room for improvement.

Critical strengths
  1. The problem is important, and the method demonstrates meaningful gains. Cross-language code matching is vulnerable to transformations such as identifier renaming. Supplementing source-code information with representations that depend less on identifiers is a reasonable and practically relevant direction. Table 6 demonstrates improvements under several cumulative obfuscation conditions. Section 4.6 further evaluates real-world obfuscation tools. These evaluations strengthen the robustness findings.

  2. The paper contributes a new dataset. XL-Asm pairs multilingual source code with virtual assembly, providing a useful resource for studying cross-language representations and robustness to obfuscation.

Critical weaknesses
  1. The methodological novelty is relatively limited, and the existing evidence does not sufficiently establish the distinctive value of virtual assembly over other complementary views.

As described in Section 3, source-to-virtual-assembly conversion relies on ViC, both branches use existing encoders, training employs standard supervised contrastive learning, and the final combination uses linear score fusion. ViC has already explored using generative virtual compilation to construct assembly data and support retrieval. The main distinction here is its application to obfuscation-robust cross-language code clone detection while retaining a source-code branch for fusion.

I acknowledge the value of this task extension and system integration. However, the adaptations in supervision, tokenization, and function-level granularity emphasized by the paper currently appear primarily to transfer existing components to a new task, rather than establish a novel mechanism with broad applicability.

  1. The “headroom principle” has empirical value, but its additional explanatory power as an independent conceptual contribution remains insufficient.

The idea that fusion is more likely to help when the source model makes errors and another model provides complementary information is highly intuitive. The principle does not demonstrate additional predictive power over existing approaches. In Table 33, the two weakest source-code encoders receive zero weight. These rows show that the assembly branch can replace weaker source models, but provide limited support for complementarity between the two views.

Questions for authors
  1. what do the authors consider the most important new technical contribution? Which existing experiment most directly demonstrates that this contribution addresses a problem that a straightforward adaptation of existing methods cannot resolve?

  2. Have the authors compared VANISH with dual-branch fusion of original source code and identifier-normalized code? Table 7 shows that using normalized code alone harms clean performance, but does this conclusion still hold when the original source-code branch is retained? This comparison would help establish the distinctive contribution of virtual assembly.

  3. In Table 37, the source-branch X-MRR scores for E2, E3, and E4 are 0.663, 0.662, and 0.666, respectively, indicating very similar headroom. Yet fusion decreases performance for E2 and E3 while improving it for E4. Could the authors specify which measurable conditions determine whether the assembly branch provides a benefit when headroom is similar?

7 comments
AuthorsOpenReview ↗

Response to Reviewers

Thank you for the suggestions. We compare assembly with cheaper views (3 backbones × 3 views; 3 seeds). CIs: 95% paired task-cluster bootstrap (2,000 resamples), uncorrected for multiplicity.

W1 & Q1: Technical contribution

To our knowledge, VANISH first fuses virtual-assembly/source features for obfuscation-robust cross-language clone detection, integrating XL-Asm (29,324 groups, 482,553 source–assembly records, 12 languages), toolchain-free virtual compilation, and trained source–assembly fusion.

All 110 renamed programs recompiled with canonicalized LLVM IR and native assembly byte-identical to originals. Within-view changes use 49 common accepted-view queries; fusion gains use 55 full-gallery queries with source fallback.

ResultX-MRR Δ (pp)95% CI
Source under renaming−22.47[−34.61, −11.54]
Assembly under renaming−1.36[−9.64, +7.79] (ns)
Fusion vs. source, renaming+4.51[+1.53, +8.42]
Fusion vs. source, clean variants+0.05[−2.80, +3.01] (ns)

Renaming exposes source-only adaptation weakness: source degrades, assembly stays comparatively stable, and fusion helps significantly. Clean gains on 15,711 records across three independent seeds are +0.43/+0.30/+0.39 pp, all CIs excluding zero; significance after program deduplication is not established (Appendix H.3). Our contribution is toolchain-free assembly integration validated under realistic, semantics-preserving obfuscation—not a new fusion operator.

W2 & Q2: Assembly-specific benefit?

Following Table 33, normalization controls retain source; α=0 denotes auxiliary-view replacement, not fusion. Controls score-fuse frozen FP32 embeddings on 15,711 records/3,728 tasks. Source weight α ∈ {0, 0.1, …, 1} is selected only on full clean dev data; ties favor the base backbone. Gains: X-MRR pp over source-only; all CodeBERT/C4 cells significant.

Backbone+ Assembly+ Normalized source+ Second source seed
CodeSage+0.43 [+0.11, +0.78]−0.22 [−0.53, +0.09] (ns)+1.19 [+0.84, +1.54]
Adapted CodeBERT+3.01+8.99+21.19 (α=0)
Adapted C4+4.78+14.11+27.96 (α=0.1)

Every base uses a separately trained CodeSage auxiliary seed: weaker-backbone comparisons change capacity; this control has more retrieval-encoder parameters than source + CLAP.

Retaining source changes Table 7: normalization alone hurts clean performance; source + normalization is competitive. CodeSage assembly−normalization: +0.66 [+0.28, +1.07] (seed 719), +0.05/+0.14 (both ns); no overall ordering. Weaker backbones reverse to −5.99/−9.33. Second-source seeds consistently beat assembly on clean CodeSage: assembly−second-seed contrasts are −0.76 [−1.17, −0.33], −1.05 [−1.41, −0.67], −1.17 [−1.56, −0.76]. CodeBERT + seed 720 is replacement (α=0; standalone 0.7295); CodeSage fuses (α=0.4).

Under verified renaming, normalization/assembly/second-seed gains are +5.57 [+0.74, +11.24]/+4.51/+6.78 [−0.21, +15.76] (ns). Normalization leads numerically in all four common/full-fallback attack cohorts; assembly−normalization contrasts are nonsignificant: renaming −0.75 [−6.66, +5.86]/−1.10 [−6.55, +4.55]; LLM rewriting −1.19 [−8.98, +5.72]/−5.03 [−11.91, +1.40].

Gains rise with headroom (1−MRR_source) for all three views; the shared baseline describes second-view opportunity, not assembly specificity. Assembly improves source-only robustness under verified renaming; normalization gives comparable measured gains at lower cost.

Q3: Gain sign at equal headroom?

Our prior explanation was inconsistent: E4 has highest source X-MRR (0.6660 vs. E2/E3’s 0.6628/0.6616), so source strength cannot explain differing gain signs. We analyze rescue/harm on 768 rebuilt records/154 problems, α=0.6. Net gain = rescue−harm; rescue = mean[max(RR_fused−RR_source, 0)]. Gains/rescue/harm: X-MRR pp.

VariantNet gain95% CIRescueHarmHarm rateError φ
E2−3.92[−6.19, −1.66]4.998.9122.4%0.153
E3−2.54[−5.16, −0.001]6.619.1421.9%0.159
E4+0.90[−1.23, +2.96] (ns)6.665.7615.1%0.179

Harm primarily explains the sign: E3/E4 rescue similarly; E4 harms less. Source-top-1 queries cannot be rescued; E2/E3/E4 harm there is 14.83/15.86/10.16. Equal headroom can yield different outcomes: correct-ranking demotions matter; decorrelation is insufficient (E4 has highest φ). This is post hoc: rescue/harm need test labels, E4’s CI includes zero, and inference-time source/auxiliary-margin prediction is untested. Historical checkpoints carry training/selection exposure (Appendix H.2); rebuilt results are not bit-exact (reproduced_within_1e-4 = false), with assembly discrepancies of order 10⁻⁴.

Paper revisions

We clarified scope, added matched controls/3-seed replication (H.3–H.5), and revised source-strength claims via rescue/harm analysis (H.8).

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer,

As we pass the midpoint of the rebuttal period, we would like to kindly follow up on our response. We have addressed your comments and uploaded a revised PDF, with changes highlighted in blue for ease of review.

We would greatly appreciate hearing whether our clarifications and additional experiments have addressed your concerns. If any questions remain, we would be happy to discuss them further. If you find that the revisions and new evidence satisfactorily resolve your main concerns, we would be grateful if you would consider raising your score.

Thank you for your time and thoughtful feedback.

Thank you for the detailed rebuttal. The new comparisons address most of my questions about the evaluation. But I’m still not convinced that the contribution is strong enough.

I want to raise an additional question before I can change the score:

As LLMs make it easier to generate functionally equivalent code across languages, what practical problem is VANISH meant to solve? Is the task (X-CCD) still meaningful? Do the ground-truth positive labels (clones) really reflect a consensus in the community? If the goal is translation detection, can VANISH find a known LLM-generated translation among independently written solutions to the same problem?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.