acceptodds
Under review as a conference paper at ICLR 2027

CryptHarness: A Harness for Agent-Assisted Cryptanalytic Research

Abstract

Cryptanalysis typically requires both a mathematical model and code, making it a natural testbed for research agents. We present _CryptHarness_, a command-line agent for cryptanalytic research, built from reusable domain skills. Given only a researcher's intuition for how a cryptographic scheme might be attacked, CryptHarness develops the attack and produces a paper-style report. The report comes with three artifacts. First, a mathematical model of the underlying problem to support a theoretical analysis of the attack. Second, a proof of concept implementation designed for readability by abstracting away nonessential details. Third, an implementation of the scheme exactly according to the specification, and of the attack itself, allowing rigorous evaluation across experiments. As a case study, we apply CryptHarness to public-key decompression for ML-DSA, the new primary post-quantum signature standard. Given only the initial idea and minimal guidance, our agent reduced the required number of signatures of the state-of-the-art decompression method by a factor of three and provided a mathematical analysis. In a separate verification layer, we formally verify a central finite-size first-moment theorem in Lean 4, providing an (almost) matching lower bound on the number of signatures required to decompress an ML-DSA-44 public key. In a second case study, we apply CryptHarness to secret-key recovery from randomness leakage in Falcon, a secondary post-quantum signature standard. The agent identified three improvements to the state-of-the-art attack, reducing the number of signatures required for key recovery by factors of 2 and 3.7 for the two existing parameter sets, respectively.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Cryptanalysis, AI-assisted research, Mathematical reasoning, Post-quantum cryptography, Research agents

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_54AQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents CryptHarness, an agent-based cryptanalytic research framework where a human expert can interact with an LLM agent (e.g., ChatGPT or DeepSeek), supporting mathematical formalization and modeling of the cryptanalysis problem, proof-of-concept attack implementation, and machine-assisted proofs (based on lean) of mathematical claims for establishing the attack. To show the effectiveness of CryptHarness, the authors analyzed two NIST-standard digital signatures schemes, ML-DSA and Falcon. For the former, CryptHarness discovered a new result on decompressing the public key using only almost one third of the signatures required from the previous SOTA. For the latter, CryptHarness devises a new secret key recovery attack under the randomness leakage uisong only a half of the signatures required from the previous SOTA.

Critical strengths
  • The proposed CryptHarness provides a clean framework where human expert can interact with an Agent in an end-to-end cryptanalysis research with machine-assisted verification, though the authors provide none of prompt inputs and produced artifacts (e.g., lean verification script or attack code).

  • The paper presents new cryptanalysis results of NIST-standard digital signature schemes (ML-DSA, Falcon) by reducing the number of valid sigatures collected by the adversary. One caveat is that the paper did not claim CryptHarness fully break these scheme; instead, it accelerates finding some necessary parts for side-channel attacks, e.g., public key decompression in ML-DSA, or leverage the attack results for secret key recovery (in the case of Falcon).

Critical weaknesses
  • I'm not fully convinced whether the paper's core results are derived solely from CryptHarness. It is already known that ChatGPT 5.6 Sol--one of the backend LLMs used by the authors--has enough capability to solve research-grade cryptographic problems with human expert's feedback. Right after the release of ChatGPT 5.6 Sol, many cryptographers solved several open problems in theoretical cryptography with its assistance [Ragavan26, Hhan26, CLX26]. I have a doubt that the claimed results could be obtained without CryptHarness.

References

[Ragavan26] Ragavan, Seyoon. "Efficient Unclonable Encryption from Pauli Eigenstates." Cryptology ePrint Archive (2026).

[Hhan26] Hhan, Minki. "Finding a Shortest Vector and More in $2^{n/2+ o (n)} $ Time using $ q $-ary Coset Difference Tree." arXiv preprint arXiv:2609.02764 (2026).

[CLX26] Coladangelo, Andrea, Liu, Qipeng, and Xie, Ziyi. "Unclonable encryption from BB84 states: a simultaneous Goldreich-Levin reduction." arXiv preprint arXiv:2608.17629 (2026).

Questions for authors
  • What happens if you directly provide the input specified in Section 3.2 and case studies to ChatGPT 5.6 Sol? As you mentioned in Section 6, its safety guardrail would not be triggered if you provide a mathematically equivalent form of the cryptanalytic question.

  • If ChatGPT 5.6 Sol alone can solve the same problem without CryptHarness as I concerned, it seems that the key advantage of CryptHarness then becomes reducing the human expert's labor in a "verifiable" way. Could you comment on assessing such a kind of advantage, or at least could you provide the exact input/output of CryptHarness in case studies? I could not assess how CryptHarness is useful for reducing the human expert's burden from the current form of the manuscript.

  • From the case study about ML-DSA, I feel that the lower bound claim of the required number of signatures for recovering t0t_{0} should be more carefully stated. Are the underling hypotheses (H1--H3) stated in Appendix C well-justified? I couldn't decide whether these assumptions are reasonble or not. In addition, does such a bound apply to other possible types of attacks? From my reading, the claimed bound is derived from Oliveira et al.'s formulation based on a system of linear inequalities, and perhaps a better attack may break this bound in the future (I'm not fully certain about this though). Current claims in the main text seem easy to misinterpret, when answers for both questions are negative, though the author acknowledged the adoption of such hypotheses as limitation.

  • The case studies give me an impression that CryptHarness merely optimizes the SOTA attack framework via human expert's initial hints, instead of developing a completely new method. I don't think this is the inherit limitation of CryptHarness because finding such a solution would be the shortest path in current experimental design. What happens if you does not provide additional contexts beyond the problem description, such as previous SOTA papers and human's hint? Can CryptHarness can discover novel attack framewoks in such a setting?

2 comments

We thank the reviewer for the thoughtful questions and for the positive assessment of the framework and cryptanalytic results. We agree that the original evaluation did not sufficiently isolate the contribution of CryptHarness from that of the underlying model. To address this we conducted additional experiments for the revised manuscript.

Comparison with ChatGPT 5.6 Sol in Codex. We agree that frontier models can contribute to research-level cryptography and have added the three works highlighted by the reviewer to our discussion of concurrent work. We added a matched generic Codex baseline using ChatGPT 5.6 Sol with high reasoning and the same problem materials, literature, expert direction, and tool access (cf. Section 6). Generic Codex recovered 3/3 ML-DSA-44 vectors at 100,000 signatures, compared with 10/10 at a mean of 60,112 for CryptHarness. For Falcon-512, generic Codex recovered 4/5 complete keys at 100 signatures, compared with 12/12 at 50 signatures for CryptHarness. Thus, the recovery techniques developed by generic Codex still succeeded in improving on the state-of-the-art but required significantly more signatures for recovery than CryptHarness.

Removing human hints. CryptHarness also succeeded without the expert attack direction (Section 6): 12/12 ML-DSA-44 vectors at a mean of 95,658 signatures and 19/20 Falcon-512 keys at a mean of 81.1 signatures among successes. In these runs, CryptHarness found methods different from those used in the original case studies: staged LP localization followed by active-set min-conflicts for ML-DSA, and scale-ray enumeration with bounded public-key-guided repair for Falcon. Their weaker performance indicates that expert guidance remained useful in our case studies.

Case-study inputs and outputs. We added the problem statements given to CryptHarness to Appendix F and provide the resulting attack code and standalone proofs of concept, in the updated supplementary material. For ML-DSA, success means recovering the entire omitted vector t0\mathbf t_0. For Falcon, it means returning a complete (f,g,F,G)(f,g,F,G) key that satisfies the public algebraic relations and produces a fresh valid signature.

Theoretical scope. We have clarified wording that could have suggested a stronger result and now describe it throughout as a conditional first-moment threshold. H1 reflects ML-DSA's fixed-weight challenge sampler, H2 uses the standard uniform-residual approximation, and H3 models independently generated signatures. The analysis was reviewed by cryptanalysts, and the predicted thresholds are within approximately 5--11% of the empirical mean recovery counts across all three parameter sets. This agreement gives us confidence that the approximation captures the main behavior observed in these experiments. We have revised the paper to state its conditional scope explicitly and avoid implying a universal lower bound.

We appreciate the reviewer's feedback, which led to a stronger evaluation and a more precise statement of the theoretical result.

Thank you for your detailed response.

The supplementary material was especially useful as it shows how simple the initial prompts can be, and the author's revision looks appropriate for me to resolve my concerns.

All of my concerns are resolved, but it still seems that the expert's guidance played a dominant role in obtaining new results (as shown in the author's response), and it would be nice if the authors could clearly present the human experts' interaction/guidance during the whole process of the cryptanalysis.

Nevertheless, I believe human intervention can be reduced with more advances in AI agents in the future, and I think this work has merit for demonstrating the feasibility of cryptanalysis research with AI agents.

I mildly support the acceptance of this work, raising my score to weak acceptance.

Reviewer_xwy6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CryptHarness is a skill-based framework for agent-assisted cryptanalytic research, combining mathematical formulation, attack development, implementation, conformance checks, and selected formal verification. Two case studies demonstrate improvements over existing attacks: ML-DSA public-key decompression through discrete tabu repair, and Falcon key recovery under exact randomness-sign leakage through joint estimation and modular candidate search. The paper also develops a conditional first-moment analysis for ML-DSA and reports a Lean formalization of its ML-DSA-44 result.

Critical strengths

Concrete research outcomes. The paper goes beyond reproducing known attacks. Table 1 reports substantial reductions in signatures for all three ML-DSA parameter sets, while the Falcon experiments report recovery of 12/12 keys at 50 and 30 signatures for Falcon-512 and Falcon-1024, respectively. If independently reproducible, these are useful domain contributions.

Attention to meaningful validation. Separating mathematical models, readable demonstrations, and standards-faithful experimental implementations is sensible. The distinction between satisfying an optimization objective and completing key recovery is particularly valuable. The disclosed correction from recovering one ML-DSA component to recovering the entire vector illustrates why these checks matter.

Critical weaknesses

Major weakness: The evaluation does not isolate the benefit of CryptHarness.

The central contribution is presented as a reusable research framework, but the evidence consists of two expert-guided investigations. There is no comparison against the same underlying model using a generic coding harness, matched literature, equivalent expert hints, and a comparable budget. There are also no skill ablations or repeated research runs. Consequently, the results establish that useful discoveries occurred in this workflow, but do not establish which parts of the harness enabled them or how reliably the workflow succeeds. This matters because the experts supplied the key direction of exploiting discreteness and public algebraic constraints, and guided subsequent exploration.

The paper also says skills were derived from research traces and refined with researchers. It should clarify whether the evaluated cases influenced those skills.

Basically, what is missing is a controlled comparison on at least these tasks, disclose when skills were frozen, and document human interventions, model roles, failed attempts, and total research costs. A large benchmark is not essential, but some evidence separating the framework from the underlying model and expert guidance is needed.

Questions for authors

Were the skills fixed before these investigations, and can you provide complete research trajectories with human interventions identified? How do a matched generic-agent baseline and the principal algorithmic ablations perform?

2 comments

We thank the reviewer for the careful assessment and for recognizing the value of the cryptanalytic results and their validation. We agree that the original evaluation did not sufficiently separate the contribution of CryptHarness from that of the underlying LLM and expert guidance. To address this we added a matched generic Codex baseline and several ablations.

Matched baseline (Section 6). Using ChatGPT 5.6 Sol with high reasoning and the same problem materials, literature, expert direction, and tool access, generic Codex recovered 3/3 ML-DSA-44 vectors at 100,000 signatures, compared with 10/10 at a mean of 60,112 for CryptHarness. For Falcon-512, generic Codex recovered 4/5 complete keys at 100 signatures, compared with 12/12 at 50 signatures for CryptHarness. Thus, the recovery techniques developed by generic Codex required overall more signatures for recovery, and conducted less experiments, i.e. provided less evidence.

Skill ablations (Section 6). Without expert direction, CryptHarness recovered 12/12 ML-DSA-44 vectors at a mean of 95,658 signatures and 19/20 Falcon-512 keys at a mean of 81.1 signatures. In these runs, CryptHarness found methods different from those used in the original case studies: staged LP localization followed by active-set min-conflicts for ML-DSA, and scale-ray enumeration with bounded public-key-guided repair for Falcon. Without the attack-ideation skill, a differentiable interval objective with gradient-guided integer repair recovered 8/12 ML-DSA-44 vectors at 70,000 signatures. The remaining skill ablations are reported in the new Section 6.

Skill development. We agree that the original submission did not make the provenance of the skills sufficiently clear. We did not develop or refine the skills using the ML-DSA or Falcon case studies, i.e. all skills were frozen before these case study experiments. Instead, we developed and refined the skills over several months on unrelated open problems in cryptography and mathematics, including problems from an external cryptanalysis competition where an earlier version of CryptHarness supported the winning entry. (We omit the competition's name to preserve double-blind anonymity and will include its details in the camera-ready version.)

We appreciate the reviewer's suggestions, which led to a stronger evaluation and clearer presentation.

I thank the authors for their detailed response and the substantial additions to the evaluation. The new generic Codex baseline, skill ablations, and clarification of skill provenance directly address my main concerns.

In particular, I find the experiments without expert direction valuable, as they demonstrate that CryptHarness can develop alternative effective attack strategies without the initial expert hints. The comparison against generic Codex also provides initial evidence that the framework contributes beyond the capabilities of the underlying language model.

However, several limitations remain. First, the evaluation does not report independent repetitions of the research process; the reported success rates concern cryptographic instances rather than independently conducted agent research trajectories. Second, it is unclear whether the generic-agent comparison uses matched research budgets, including model calls, computational resources, and human interventions. I would also appreciate clarification of the runtime and model configurations, given the descriptions of Pi and DeepSeek in Section 3 versus Codex in Section 6. Third, the different sample sizes and reported signature counts make the magnitude of the improvement over generic Codex difficult to quantify precisely.

Thus, while the revised experiments provide promising evidence for CryptHarness, they do not yet establish its reliability or generalizability as a research framework.

Nevertheless, I acknowledge that the authors have meaningfully strengthened the paper and addressed the main direction of my criticism. Given the combination of concrete cryptanalytic contributions and the improved framework evaluation, I increase my rating to weak acceptance.

Reviewer_dLo9OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper explores the idea of using AI agents for cryptanalysis. A command line agent, CryptHarness, is built from domain skills to aid in cryptanalytic research. The agent is guided by a human researcher’s intuition for how a cryptographic scheme might be attacked. It then develops the attack and produces a paper-style report. The report comes with three artifacts. First, a mathematical model of the underlying problem to support a theoretical analysis of the attack. Second, a proof of concept implementation designed for readability by abstracting away nonessential details. Third, an implementation of the scheme exactly according to the specification, and of the attack itself, allowing evaluation across experiments. The paper provides examples of effectiveness of using CryptHarness on two PQ algorithms. First, working on public-key decompression for ML-DSA, the new primary post-quantum signature standard, and second secret-key recovery from randomness leakage in Falcon. In both cases, given only the initial idea and minimal guidance, the improved on state-of-the-art attacks.

Critical strengths

The cryptanalysis improvements in the examples are nice, but the method is the most impactful contribution. It is still early work, proof-of-concept by the the implications for the field is significant.

The presentation of the idea, and the rigour of validating the results is excellent.

Critical weaknesses

None.

Questions for authors

The minor drawback of the paper is that one can always show one or two examples of something working. Here the examples have significant impact in themselves but in your opinion how generalisable would such an approach be? Also, what is expected scope of an agent - you mention reusable domain knowledge - will we have one big cryptanalysis agent or better served having agents focused on specific target domains?

Future work: In would be interesting to see a more wide ranging study of agent abilities across historical and future crypto algorithms (underlying approaches). So expanding the to a larger number of case studies or even setting an initial benchmark across major crypto math domains for agent cryptanalysis.

2 comments

We sincerely thank the reviewer for the encouraging assessment, particularly the positive evaluation of the framework and the rigor of our validation.

We expect the most useful design to combine a shared set of research skills with mathematical and verification tools specialized to the particular cryptographic domains. CryptHarness skills such as translating specifications into mathematics, testing candidate attacks, and producing inspectable implementations can transfer across domains, while the relevant representations and tools will often need to be specialized and adapted by experts.

We also agree that a broader study across cryptographic primitives and mathematical structures would help establish how widely the approach transfers. We have added this direction to the discussion, including the reviewer's suggestion of a benchmark spanning cryptographic families and attack types to study which parts of the workflow transfer and where specialized tools are most useful.

Noted, thank you for the response. I maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.