acceptodds
Under review as a conference paper at ICLR 2027

Physical survival and provenance limits on watermarking AI-generated crystals

Abstract

When crystals are generated, they are relaxed and standardized before scientific use. This processing can erase or alter structural evidence of how they were produced. We ask what a structure-only watermark can preserve through this process, how an auditor should calibrate the surviving evidence, and which provenance claims the resulting decision supports. BasinMark connects a deposited structure to a collection-level audit by selecting generated candidates according to keyed structural fingerprints. Exact relaxation to the same basin minimum erases coordinate marks, so the protocol selects among alternatives whose fingerprint distinctions may survive deposition. Across the evaluated generators and interatomic potentials, cell-shape bins account for much of the observed fingerprint instability. Removing these bins in two generator populations substantially improves fingerprint-label survival while preserving structural diversity. Under a key-independent null, counting each unique fingerprint class once provides exact false-positive control. Detection operates at the structural-class level, however, so collections with identical fingerprints remain indistinguishable. Watermark evidence can therefore be copied or reused across datasets. Physical processing, fingerprint design, and the handling of structural duplicates determine how much watermark evidence survives and what that evidence can establish.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
watermarking, crystal generation, materials generative models, provenance, relaxation robustness, fingerprint auditing, collision-aware testing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ap3GOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper proposed a candidate-selection protocol, BasinMark, for structure-alone provenance auditing of AI generated crystals following relaxation and standardization. BasinMark selects generated candidates according to keyed structural fingerprints and audits sets of crystal structures by counting each unique fingerprint class only once.

The paper shows that exact same-basin relaxation may erase coordinate perturbations and removing cell-shape bins may improve fingerprint-label survival with no distortion of distinct class labels in two benchmark datasets. Under a key-independent null, the deduplicated class auditing provides proper false-positive control. In addition, collections with identical fingerprints remain indistinguishable, which allows watermark evidence to be copied or reused across datasets while limiting provenance claims to fingerprint-class resolution.

Critical strengths

Studying the sets of crystal structures after relaxation and crystallographic standardization, instead of using only the AI generator's raw output, is of practical relevance. Removing cell-shape bins substantially improves watermark survival through relaxation without distorting distinct fingerprint class labels. Proposition 2 explains why duplicate fingerprints do not provide fresh independent evidence under a key-independent collection and ideal random-function labels. The naive 6.4% rejection rate for WyckoffLLM-vanilla illustrates the practical consequences of ignoring collisions.

Critical weaknesses

The entirety of the proposed watermarking pipeline is not tested together. In effect, the paper uses separate experiments to evaluate fingerprint survival and detection, in which the joint post-deposition power remains unknown. Thus, the reported detection rates are not collectively reflective of the reliability of BasinMark in detecting watermarks after the full generation, selection, and deposition workflow. There lacks supportive evidence for the central practical claim in this paper.

One noticeable weakness is that there exist inconsistencies between implementation and simulation designs in empirical validation, ranging from the released code and available detailed results in GitHub. (i) The released reuse simulation uses the secret key to retain only green fingerprints before constructing the adversary’s observations. This differs from the stated threat model. The code is absent to generate results of attack rates reported in Figure 2e. (ii) The WyCryst negative controls available in the GitHub report 1% rejections for both unmarked and shuffled-key collections at (m=8) at a nominal 0.1% threshold, which contradicts to the statement that unmarked-key false-positive rates remain at most 0.5%. (iii) Executing the released function on seven equally populated classes returns 7.75, which exceeds the theoretical bound of 7. This is an inconsistent reporting with respect to the bound neff≤Kn_{\mathrm{eff}}\leq K. This mismatch complicates the interpretation of the diversity results.

Questions for authors
  1. Algorithm 1 relaxes candidates before watermark selection, while the survival experiments examine fingerprint changes across relaxation, and the limitations section states that joint post-deposition power remains unmeasured. Such mixed messages make the paper hard to follow; it is unclear whether the reported detection rates validate fully the intended workflow. Confusion arises: What processing occurs after selection, and which results demonstrate detection after that processing?

  2. The released reuse routine uses the secret key to retain only green fingerprints before constructing the observations. Genuine selected outputs can include non-green fallback selections, so this filtering evaluates an easier attack than observing actual outputs without the key. Which script produced Figure 2e? Did those experiments retain fallback outputs without filtering observations by their true green status? The original script and per-trial results are absent so it is impossible to verify the reported attack rates.

  3. The WyCryst results available in GitHub show 2/200 (1%) rejections for both unmarked and shuffled-key controls at (m=8) whereas S2.7 states that unmarked-key rates remain at most 0.5%. Such summary available in the GitHub discounts the anomaly using a pooled multiple-testing correction, which in fact addresses a different question from per-audit false-positive calibration. How do you reconcile these results? What calibration evidence accounts for the reused keys and overlapping collections?

  4. Proposition 2 defines neff=(∑iwi2)−1)n_{\mathrm{eff}}=(\sum_i w_i^2)^{-1}), but the implementation reports a bias-corrected population collision estimator. For seven equally populated classes, it returns 7.75 exceeding the theoretical bound of 7 from neff≤Kn_{\mathrm{eff}}\leq K if deemed as the quantity from Proposition 2. Which quantity is intended in the empirical results? Would using the proposition’s definition change any diversity or feasibility conclusions?

5 comments

Thank you for checking the implementation against the paper. Each of the three discrepancies the reviewer found has a concrete fix in the revised code, data and text, and the end-to-end question now has a measured answer.

In W1 and Q1 the reviewer was concerned that the submission measured survival and detection separately, so it was unclear what processing happens after selection and which results show detection after it. The revision measures them together. Selection uses producer coordinates; the auditor sees only independently processed deposits. Section 4.4 and Table 1 report that chain: the producer selects 32 of 256 relaxed candidates, a depositor independently re-relaxes those coordinates (variable-cell CHGNet-r2SCAN), standardizes the primitive cell and writes a five-decimal CIF, and the auditor reads the files. Detection is 499 of 500 on MatterGen and 476 of 500 on CrystalFormer, against 491 and 12 for fixed bounded batches. On two held-out CrystalFormer cohorts that share no classes with earlier data, under a frozen protocol, ordinary processing gives 498 of 500 on both. A 1% strain before re-relaxation gives 492 and 493, and 496 and 497 with an optional producer preview. Every result in the paper is labeled by its input geometry and endpoint. The archived MatterGen DFT comparison (495 of 500) is labeled on its own because the starting coordinates of those DFT runs were not recorded.

In W2(i) and Q2 the reviewer found that the released reuse routine used the secret key to keep only green fingerprints before building the observer's inputs, an easier attack than the stated threat model, and that the script and records behind Figure 2e were missing. Both are fixed. The filtering routine is gone. The replacement driver gives the observer every emitted record in order, including non-green fallbacks, and only its public fingerprint. Only the producer and the auditor get the key. At ten observations the observer saw 2,000 emissions per generator, with six MatterGen and three CrystalFormer fallbacks, each in a different trial. Replay-allowed collections rejected in 194 and 197 of 200 trials, and the nine fallback trials account for all nine misses. Excluding exposed record IDs gives 0 of 200 for both. Figure 2e and Table 8 are rebuilt from these trials. The per-trial observation records (ordered IDs, fallback positions, decisions) and the plotting script are in the anonymous archive.

W2(ii) and Q3 pointed out that the WyCryst controls in the repository reject 2 of 200 (1.0%) at m = 8 while Appendix S2.7 stated at most 0.5%, and that the pooled multiple-testing correction answers a different question than per-audit calibration. We agree on both points. The text now reports the 2 of 200 counts for each control, and we no longer use pooled corrections to describe per-audit calibration. Fresh controls that hold the batches fixed and draw two new keys per trial give 15 of 20,000 for unmarked minimum-energy selection and 10 of 20,000 for an independent key on marked outputs, against an attainable discrete null of 0.054% at the nominal 0.1% level. Where keys or collections are shared, the paper states the pairing and reports uncertainty over the independent units (Appendix S2.7).

In W2(iii) and Q4 the reviewer noticed that the released function returns 7.75 for seven equally populated classes, above the bound n_eff <= K = 7 of Proposition 3 (Proposition 2 in the submission). The function returned a bias-corrected population estimator under that name. The API now reports the observed quantity of Proposition 3 as n_eff and the population estimator under a separate name, with regression tests for balanced and duplicate-heavy samples. The recomputed observed values are 1,490.1 for 1,495 MatterGen classes and 981.1 for 985 CrystalFormer classes, so the diversity conclusions stand. Rejection decisions use the distinct-class count K and were never affected. The seven-class example now sits next to Proposition 3.

Because the selectors of Section 4.4 differ from the fixed batches of Algorithm 1, Appendix S3.2 adds Proposition A, which writes the fixed-batch green count as a Poisson-binomial over the affordable classes in each block. With no fitted parameter it predicts producer-stage power of 3.8%, 1.9%, 3.2% and 98.3% on four archived cohorts, against observed 2.8%, 1.6%, 4.4% and 99.0%. Proposition B covers the pooled selector; its supply condition held in all 2,000 trials. Proposition C states the two-view audit calibration with a proof, checked against exhaustive enumeration in the test suite.

The reviewer's code checks made the empirical claims easier to verify. Every discrepancy identified is resolved in the released code and data, and the main claim now has a direct end-to-end measurement. We believe the revision answers each point with measurements in the paper, and we respectfully ask the reviewer to reconsider overall score by increasing.

The revision provides additional evidence on detection after downstream processing. However, the revised framework makes substantive changes to the original selection procedure, and the motivation and consequences of these changes need further clarification.

The original method selects the lowest-energy green candidate within each batch, falling back to the batch minimum only when no green candidate exists. The revision introduces an energy cap, which can exclude existing green candidates, and then introduces pooled selection across batches. Adding independent downstream processing does not itself require either change. The revision lacks the evidence for the unrestricted selector alongside the capped and pooled selectors, and insufficient explanation of the detection–quality trade-off that motivates each change.

Another weakness concerns the implementation of pooled selection. There is a clear discrepancy between the revised pooled selection (selecting distinct green fingerprint classes), and the implementation (selecting individual candidate records). Selecting candidates is consistent with the original method, and the final audit deduplicates their fingerprints. The revision is unclear on whether the pooled-selection description assumes distinct candidate fingerprints and how duplicate classes are handled when that assumption does not hold.

In conclusion, the fundamental weakness of the proposed method is not well addressed.

We thank the reviewer for the response. The matched evidence below addresses both the unrestricted baseline and whether the record/class discrepancy affects the reported experiments.

Independent downstream processing does not require an energy cap or pooling. These are additional design choices motivated by selection quality: the cap limits selection energy, while pooling allows interchangeable output slots to share candidates under energy constraints. Their consequences should be evaluated against the original unrestricted rule.

The following archived comparison holds the no-cell fingerprint, marking probability gamma = 1/2, candidate draws, keys, downstream processing, and deduplicated binomial auditor fixed, with nominal alpha = 0.001. Each trial selects 32 structures from 256 candidates. The downstream procedure independently re-relaxes the selected coordinates with CHGNet-r2SCAN, standardizes the primitive cell, and writes and rereads a five-decimal CIF. Using gamma = 1/2 throughout isolates the selection changes from the revised default gamma = 5/8.

GeneratorSelectorDetection after processingExcess energy (meV/atom)Stable yield
MatterGenOriginal unrestricted fixed batches500/50032.174.0%
MatterGenFixed capped batches491/50021.877.0%
MatterGenPooled, slot and mean caps500/50016.286.3%
CrystalFormerOriginal unrestricted fixed batches500/500400.022.7%
CrystalFormerFixed capped batches12/50056.731.8%
CrystalFormerPooled, slot and mean caps455/50049.330.7%

"Original" denotes the submitted unrestricted selection rule, evaluated with the same no-cell fingerprint as the other rows to isolate selection effects. Excess energy is the producer ranking energy relative to the cheapest 32 candidates in each trial. Stable yield uses archived DFT hull energy <=0.10 eV/atom for MatterGen and archived phonon-stability labels for CrystalFormer, divided by all emissions. These source/reference labels are withheld during selection; they are not new stability tests of the deposited structures. The 500 trials vary keys and candidate draws within each fixed cohort.

The original selection rule retains detection under this tested processing: 500/500 for both generators. The CrystalFormer 12/500 result belongs to the capped variant. Capping reduces excess energy from 400.0 to 56.7 meV/atom but sharply reduces detection; pooling recovers 455/500 detections at 49.3 meV/atom. Compared with unrestricted selection, pooling sacrifices detection while reducing ranking energy and increasing source-label stable yield from 22.7% to 30.7%. Compared with capped selection, stable yield decreases from 31.8% to 30.7%. These results support a measured detection-quality trade-off, not a uniform improvement across all metrics. We will add the unrestricted baseline and this controlled comparison to the manuscript.

Regarding duplicate handling, the implemented pooled selector maximizes green candidate-record count subject to its energy constraints. For each proposed count, it considers the cheapest required green records and non-green records. It does not enforce distinct fingerprint classes during selection. We will correct the manuscript's use of "classes" in describing this operation.

We checked both candidate cohorts underlying the table: each contains 384 records with 384 distinct producer fingerprints. Thus, the record/class distinction does not change the selections in these experiments. For general inputs with duplicate classes, multiple selected records may share a producer fingerprint. The final audit always counts distinct deposited fingerprints once; repeated deposited fingerprints provide no additional class evidence. Maximizing green records does not guarantee distinct output fingerprints or maximize distinct green classes. Any interpretation of green-emission counts as distinct-class evidence must state the relevant distinctness assumption.

The evidence therefore separates the original selector's downstream survival from the quality constraints introduced by the extensions, and establishes the scope of the implementation discrepancy. We will make the record-level selection objective and its relationship to class-level evidence explicit.

We respectfully ask you to reconsider the soundness assessment and overall score in light of this controlled comparison and the verified duplicate analysis. If you consider the fundamental weakness unresolved, please identify the specific remaining claim or missing comparison so that we can address it directly.

Reviewer_CrqxOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces BasinMark, a selection-based watermarking protocol for generated crystal structures, and examines the statistical and provenance limits of structure-only auditing. A secret key assigns labels to structural fingerprints; the producer preferentially selects candidates with designated labels, and the auditor tests for enrichment among distinct fingerprint classes. The paper shows that removing cell-shape features substantially improves fingerprint survival under relaxation in two generator populations. It also establishes a deduplicated audit under a key-independent null and demonstrates that fingerprint-class reuse can transfer detection signals without establishing individual record origin.

Critical strengths

A concrete representation improvement supported by substantial empirical evidence. Removing cell-shape features reduces fingerprint changes from 78.0% to 25.7% for MatterGen and from 67.8% to 4.0% for CrystalFormer, while retaining nearly all observed distinct classes. This identifies an actionable design choice for watermarking physically processed structural data.

Careful calibration and interpretation of provenance evidence. The paper distinguishes repeated records from independent fingerprint classes and demonstrates the consequences of counting duplicates as independent evidence. Its reuse experiments further establish that detection supports class-level association rather than unique record-level attribution. These distinctions are consequential for responsible interpretation of scientific watermarking.

Critical weaknesses

The complete watermarking pipeline is not adequately validated. Algorithm 1 relaxes candidates before watermark-based selection, whereas the principal survival experiments compare structures before and after relaxation. The deployment-relevant question is whether selected structures retain sufficient signal after subsequent processing. Neural-generator detection and fingerprint-survival experiments are conducted separately, leaving joint detection power unmeasured. This limits the evidence that BasinMark reliably addresses its motivating application.

The practical trade-offs of selection and fingerprint coarsening are incompletely characterized. Retaining aggregate distinct-class counts does not fully establish discrimination among structurally different crystals, particularly within a fixed composition. The evaluation also does not adequately quantify material-quality changes or computational costs relative to unmarked selection under a matched budget. These omissions limit assessment of the protocol’s practical utility.

Questions for authors

Which processing operations occur before watermark selection, and which occur afterward? What detection power is achieved when selected neural-generator outputs undergo the intended downstream relaxation, standardization, and quantization before auditing?

Under matched computational budgets, how does watermark-based selection affect energy, stability, validity, and structural diversity relative to unmarked selection?

Beyond aggregate distinct-class counts, what evidence shows that removing cell-shape features preserves discrimination among different structures with the same composition?

2 comments

Thank you for separating the scientific question from the fingerprint benchmark, and for your positive reading of the representation and calibration results. The revision measures what a third party actually receives and what the producer gives up to make a detectable collection.

In W1 and Q1 the reviewer was concerned that the submission never states which processing steps come before watermark selection and which come after, and never measures the detection power left once selected outputs go through the intended downstream relaxation, standardization and quantization. The revision answers both. Before selection, the generator proposes candidates and the producer relaxes them and reads their fingerprints and ranking energies. After selection, a separate depositor re-relaxes the chosen coordinates (variable-cell CHGNet-r2SCAN, 0.05 eV/A or 150 steps), standardizes the primitive cell at 0.1 A, and writes and rereads a five-decimal CIF. The auditor sees only those files. Section 4.4 reports joint power under exactly this chain. With 32 outputs from 256 candidates, 499 of 500 MatterGen and 476 of 500 CrystalFormer collections are detected; fixed bounded batches give 491 and 12. On two held-out CrystalFormer cohorts under a frozen protocol, ordinary processing gives 498 of 500 on both, and strain followed by re-relaxation gives 492 and 493, or 496 and 497 with the optional producer preview. A nonconverged endpoint contributes no class, and every attempted collection stays in the denominator. Algorithm 1 now names the producer, depositor and auditor roles.

W2 and Q2 asked how keyed selection changes energy, stability, validity and structural diversity relative to unmarked selection under the same computational budget. The new Table 1 compares, on the same 256 candidates and keys, the cheapest unmarked release, fixed bounded selection and the energy-budgeted exchange selector. Marking costs 11.9 meV/atom over the cheapest release on MatterGen and 37.3 on CrystalFormer. On MatterGen the DFT-confirmed stable share is 86.6% marked and 82.2% unmarked. On CrystalFormer the phonon-stable share is 31.9% marked and 38.8% unmarked, which we report as the cost of marking that generator. All 32 selected structures pass the minimum-distance validity check and have distinct formulas in every trial, and collections average 32.0 and 31.99 distinct fingerprints after processing. Selection on precomputed candidates takes milliseconds; the candidate budget is the main cost, at 6.0 relaxation worker-seconds per output for eight candidates against 0.87 for one.

In Q3 the reviewer asked for evidence, beyond aggregate class counts, that removing cell shape still separates different structures of the same composition. Appendix S3 now tests exactly that, on same-formula pairs that a structure matcher calls distinct. On the MP-20 test split the full fingerprint merges 18 of 684 such pairs and the no-cell fingerprint merges 101. On the validation split the counts are 20 and 93 of 752. A reference-assisted comparison of neighbor-distance ranks brings the no-cell counts down to 92 and 85 and rejects the one matcher-equivalent validation pair that shared a coarse class. So the survival gain has a measurable resolution cost within a composition, and the paper gives both numbers side by side.

The presentation and informativeness ratings led us to reorganize the main text around one line of argument. The revision adds one results subsection and one table to it, and moves the protocol details, the calibration and the selector power law to the appendix. Every change is marked in red.

The direct workflow test and the matched-budget comparison answer each question with numbers in the paper. We respectfully ask the reviewer to raise their overall score given the changes made during rebuttal.

We hope additions (added downstream detection, material-quality, and structural-resolution results) resolve your concerns and support a higher overall score, and would welcome clarification of any points that remain open.

Reviewer_XG3dOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the robustness and provenance limitations of watermarking AI-generated crystal structures. The authors propose BasinMark, a selection-based watermarking framework that preferentially selects low-energy candidates with favorable keyed structural fingerprints. The paper makes three main contributions: (1) analyzing watermark survival under physical relaxation and improving fingerprint stability by removing cell-shape features; (2) establishing statistically calibrated detection through fingerprint deduplication; and (3) demonstrating fundamental provenance limitations through fingerprint-class reuse. Experiments across several crystal generators and interatomic potentials support these findings.

Critical strengths
  1. Relevant problem and informative experimental findings. The paper addresses watermark robustness under physical post-processing, an important challenge for AI-generated scientific data. Its analysis of fingerprint instability across crystal systems and the substantial stability improvements achieved by removing cell-shape features provide useful insights for practical watermark design.

  2. Valuable theoretical analysis of watermarking limitations. The theoretical results on relaxation-induced coordinate watermark removal, fingerprint collision calibration, and provenance indistinguishability provide a systematic understanding of the limitations of structure-only watermarking, particularly the distinction between statistical watermark detection and reliable provenance authentication.

Critical weaknesses
  1. Insufficient end-to-end validation and evaluation of quality–cost trade-offs. Although Appendix S2.7 reports marked-output detection controls, the paper does not sufficiently establish end-to-end detection performance after independently processing already-watermarked structures. Algorithm 1 selects candidates after initial relaxation, whereas the main survival experiments compare fingerprints before and after that relaxation. The robustness of the complete watermarking pipeline under subsequent processing therefore remains insufficiently validated. Furthermore, selecting the lowest-energy watermark-eligible candidate rather than the lowest-energy candidate overall may affect generation quality, while generating and relaxing multiple candidates may increase computational costs compared with single-candidate generation. These trade-offs are not adequately quantified.

  2. Insufficient evaluation of adversarial robustness and practical provenance security. Although the paper explicitly demonstrates fingerprint-class reuse limitations, it does not systematically evaluate watermark robustness against active attacks. In particular, it remains unclear whether physically plausible structural modifications or alternative relaxation pipelines could weaken watermark detection while preserving material quality. Furthermore, removing cell-shape features may increase cross-database fingerprint matches and facilitate class-reuse attacks, but the impact on attack success is not adequately quantified. Finally, the false-positive guarantees under the unmarked null assume key-independent collections and do not cover repeated post-hoc key trials in provenance disputes. These limitations leave the scheme's security under such adversarial settings insufficiently characterized.

Questions for authors
  1. Could the authors clarify the scope of the end-to-end controls reported in Appendix S2.7 and provide detection-power measurements after already-watermarked structures undergo independent downstream relaxation or crystallographic standardization?

  2. How does BasinMark affect energy distributions, structural diversity, material stability, and computational cost compared with unwatermarked baselines, including both identical candidate budgets and single-candidate generation?

  3. How robust is the watermark against physically constrained structural modifications and alternative relaxation pipelines? Can the authors quantify the degradation in detection performance while assessing whether the modified structures retain acceptable material quality?

  4. Could the authors quantify how removing cell-shape information affects fingerprint collisions and cross-database overlap at larger scales? In particular, does the improved fingerprint stability increase the success rate of class-reuse attacks compared with the original fingerprint representation?

  5. Does the intended provenance threat model consider adversaries who select verification keys after observing a dataset? If so, how are repeated key trials addressed to prevent misleading provenance claims, and would a prior key commitment or other safeguards be necessary?

4 comments

Thank you for drawing the line between a detectable collection and a claim of ownership. These questions shaped the end-to-end protocol and the threat model in the revision.

In W1 and Q1 the reviewer was concerned that the controls in Appendix S2.7 do not establish detection after already-selected structures are processed by someone else. That is correct: the appendix audits the selected fingerprints at the moment of release. The new Section 4.4 closes this gap. It follows the selected coordinates through an independent variable-cell CHGNet-r2SCAN relaxation, primitive-cell standardization and a five-decimal CIF round trip, and audits only the deposited files. With 32 outputs from 256 candidates, the energy-budgeted selector detects 499 of 500 MatterGen collections and 476 of 500 CrystalFormer collections. Fixed bounded batches give 491 and 12. On two held-out CrystalFormer cohorts that share no classes with earlier data, under a protocol frozen before the results were opened, ordinary processing gives 498 of 500 on both. A 1% strain before re-relaxation gives 492 and 493 with the survival-ranked audit, and 496 and 497 with the optional producer preview. Wrong keys fire 85 and 93 times per 100,000.

W1 and Q2 also asked what marking costs against an unwatermarked baseline, both at the same candidate budget and against single-candidate generation. To answer this, Table 1 now puts the cheapest unmarked 32-of-256 release next to the marked one. Marking costs 11.9 meV/atom on MatterGen and 37.3 on CrystalFormer. The DFT-confirmed stable share on MatterGen rises from 82.2% to 86.6%. The CrystalFormer phonon-stable share falls from 38.8% to 31.9%, and the paper says so. One candidate per output gives 13.3% stable on CrystalFormer. All releases have 32 distinct formulas and pass validity. Selection takes milliseconds. Relaxing eight candidates per output costs 6.0 worker-seconds against 0.87 for one. The preview adds 256 relaxations per collection and 2.9 to 4.3 meV/atom. Producer and auditor costs are listed separately in Appendix S3.

Under W2 and Q3 the reviewer raised the possibility that an attacker weakens detection with edits that keep the material acceptable, or by using a different relaxation pipeline. We tested both cases. The attacker gets eight binary audit queries and may shear any emitted crystal within 10 meV/atom and the validity check. It erases 15 of 489 detections. If edits must keep the crystal phonon-stable at both endpoints, it erases 2. Replacing the downstream model with MACE-MPA-0 on the same released structures and keys gives 496 and 493 detections after strain and 496 and 497 after ordinary relaxation. All 432 endpoints converge and pass validity (Appendix S3, Table 10).

The second part of W2, and Q4, concern whether removing cell shape, which improves survival, also raises cross-database matches and so makes class-reuse attacks easier. It does, and the revision measures by how much. Across 9,046 MP-20 test structures and 27,000 OQMD records, the full and no-cell fingerprints share 165 and 267 classes. With 256 observed outputs, transfer succeeds in 23 and 86 of 200 trials. Excluding external records equivalent to an exposed geometry gives 0 of 200 for both. The full sweep over exposure (Appendix S3.1, Figure 3) follows P{Bin(N, s) >= 10}, where s is the catalogue-matched share of lowest-energy picks: 0.025 with cell shape and 0.040 without. Coarsening therefore shifts the curve by 1.6x, and energy preference on its own raises exposure by 1.35x. Provenance claims are stated at class resolution.

Finally, in W2 and Q5 the reviewer asked whether an adversary who chooses a key after seeing a dataset can manufacture a provenance claim. Without a safeguard it could, because the single-key guarantee needs a key fixed before inspection. The revision therefore requires a timestamped commitment to the key, the fingerprint rule, the selection rule and the threshold, revealed later for verification, and a Bonferroni per-key level of alpha/L for L declared keys (Section 3.4). A key search that was never declared cannot support the stated significance.

We respectfully ask the reviewer to increase the overall rating in light of the end-to-end validation and the quantified threat model.

We would appreciate the reviewers' follow-up on whether the end-to-end experiments and clarified provenance threat model address your concerns. Based on these results, we respectfully ask you to reconsider your overall rating and let us know if any specific issues remain unresolved.

Thank you for the detailed response and the additional experiments. The new end-to-end evaluations, quality–cost comparisons, and adversarial robustness analyses address my main concerns and substantially strengthen the paper. I appreciate the authors' efforts in revising the work and will raise my rating accordingly.

Thank you very much for taking the time to review our revisions and for raising your rating. Your constructive questions directly shaped the end-to-end protocol, quality–cost trade-off analysis, and adversarial robustness experiments, which significantly strengthened the paper. We sincerely appreciate your engagement throughout the review process.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Responses and Changes

We thank the reviewers for reading the paper closely. All three reviews ask the same main question: once someone else processes the crystals a producer selected, can the auditor still detect the mark, and what does marking cost in material quality and compute? The revision answers this with new measurements. We have uploaded the revised version of the manuscript with changes tracked in red text.

Detection after independent processing is now measured directly. The producer picks 32 structures out of 256 relaxed candidates. A separate depositor takes those coordinates, re-relaxes them with a variable-cell CHGNet-r2SCAN run, standardizes the primitive cell, and writes and rereads a five-decimal CIF. The auditor sees only the deposited files and a key fixed in advance. Every attempted collection counts in the denominator. Under this protocol a selector that spends an energy budget across interchangeable output slots detects 499 of 500 MatterGen collections and 476 of 500 CrystalFormer collections. Fixed bounded batches of eight candidates with the same energy cap detect 491 and 12. We then froze the protocol and tested two new CrystalFormer cohorts of 384 structures that share no classes with earlier data. Ordinary processing detects 498 of 500 on both. When a 1% axial strain is applied before re-relaxation, a survival-ranked audit detects 492 and 493, and 496 and 497 when the producer runs an optional stress preview. Wrong keys fire 85 and 93 times per 100,000 at the nominal 0.1% level.

Quality and cost are compared at a matched candidate budget, as two reviewers asked. Compared with the cheapest 32 of the same 256 candidates, the marked release costs 11.9 meV/atom on MatterGen and 37.3 on CrystalFormer. On MatterGen the DFT-confirmed stable share is 86.6% marked and 82.2% unmarked. On CrystalFormer the phonon-stable share is 31.9% marked and 38.8% unmarked, and the paper reports that as the cost of marking that generator. Every release has 32 distinct formulas and passes the minimum-distance validity check. Keyed selection on precomputed candidates takes milliseconds. The eight-candidate budget costs 6.0 relaxation worker-seconds per emitted structure against 0.87 for one candidate, and the optional preview adds 256 relaxations per collection. The theory now covers these selectors: fixed-batch power is a Poisson-binomial over the affordable classes in each block and predicts the observed rates with no fitted parameter.

Adversarial robustness, raised by Reviewer XG3d, is tested in three ways a provenance claim can fail. Removal: an attacker with eight yes/no audit queries and a 10 meV/atom shear budget erases 15 of 489 detections, or 2 when every edit must keep the crystal phonon-stable at both endpoints. Pipeline change: swapping CHGNet-r2SCAN for MACE-MPA-0 on the same releases still gives 496 and 493 detections after strain. Reuse: across 9,046 MP-20 and 27,000 OQMD records, the chance of transferring the signal follows a binomial law in the number of observed outputs. Removing cell shape shifts that curve by a factor of 1.6, and excluding geometry equivalent to an exposed structure gives 0 of 200 at every exposure. Against post-hoc key search, the paper requires a timestamped commitment to the key and audit rules, with a Bonferroni per-key level for any declared set of keys.

The implementation discrepancies Reviewer ap3G found are fixed in the released code and data. The reuse driver now gives the observer every emitted record, fallbacks included, and never the key, and we publish the per-trial records. The audit now reports the observed effective sample size of Proposition 3 (7 for seven equal classes) under one name and the population estimator (7.75) under another. The WyCryst controls are reported as 2 of 200 in each arm, with fresh 20,000-key controls at 15 and 10 per 20,000 against an attainable null of 0.054%. The anonymous archive holds the selector, auditor, per-trial records, hashes and replay commands. 61 tests and 156 Lean-checked theorems pass.

Readability is addressed by reorganizing the paper around one line of argument. The abstract moves from the processing problem to the mechanism to the measured outcome. The Method presents the observation channel, then fingerprint stability, then selection, calibration and resolution, in the order the Experiments test them. Each equation and theorem is followed by its consequence, every scope condition is collected in the Limitations, and the Related Work closes with the chain that links the three results.

The reviewers' questions led to one new contribution, detection after deposition, so the paper now lists four contributions instead of three. We respectfully ask each reviewer to increase their overall rating in light of the revision, and we welcome any remaining concern that we can still address in the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.