acceptodds
Under review as a conference paper at ICLR 2027

Hypothesis-Verified Neural-Symbolic Taint Analysis for Vulnerability Detection in Python Libraries

Abstract

Static taint analysis is widely used for vulnerability detection, and one of the factors that decides its precision is the treatment of sanitizers: mainstream analyzers model a sanitizer as a fixed cleansing rule whose effectiveness follows from rule existence alone. In Python, where sanitizing behavior is conditional on runtime types, control flow, and library versions, this assumption yields false negatives and false positives. We propose SanTA, a neural-symbolic method in which the conditions for a sanitizer to take effect are no longer unilaterally assumed by the analyzer but exist as hypotheses to be verified. SanTA uses a large language model to identify candidate sanitizer calls and formulate effectiveness conditions in a closed vocabulary; a symbolic layer verifies each against source code facts, and the call blocks taint only when all conditions and obligations are established, with auditable evidence on every verdict. On the CVEfixes dataset SanTA attains the best F1 (0.67) and recall (0.83) of the analyzer roster; on our conditional-sanitizer benchmark, ablations attribute the gains to the verification layer. A campaign over 30 popular open-source Python packages surfaced 15 previously-unknown vulnerabilities, 3 with assigned CVE ids.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Taint analysis;Vulnerability Detection; Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_S8AfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SanTA, a neural-symbolic framework for analyzing conditional sanitizers in Python. It uses an LLM to identify candidate sanitizer implementations and propose conditions for their effectiveness using a closed vocabulary. A mechanical verification layer then checks these conditions against program facts, inspects rejecting guards, and performs additional transformer and whitelist-rewriter checks to determine whether taint propagation should be blocked. The evaluation covers a benchmark of 50 CVEs involving conditional sanitizers, a CVEfixes pool of 163 CVEs, and an analysis of 30 Python packages. SanTA achieves an F1 score of 0.67 and recall of 0.83 on CVEfixes. The package analysis yields 15 reported findings, three of which have been assigned CVE identifiers.

Critical strengths
  • well-design technique. SanTA combines LLM-generated sanitizer hypotheses with mechanical checks against program evidence.

  • Clear presentation. The paper explains its motivation clearly and presents the framework in well-organized stages.

Critical weaknesses
  • Unclear positioning. The introduction does not clearly explain the limitations of prior work or how SanTA addresses them, making it difficult to understand the paper’s contribution.

  • Limited novelty. The method mainly combines LLM-generated sanitizer hypotheses with existing taint analysis and SMT checks, without clearly establishing a methodological advance beyond this integration.

  • Limited field validation. The real-world campaign reports only 15 findings across 30 Python packages, providing limited evidence of generalization across diverse sanitizer patterns and vulnerability types.

  • Unclear Evaluation. Table 3 reports identical results before and after the verdict layer (F1 = 0.62); the improvement to 0.67 comes entirely from additional fixed-version-specific gates whose inputs and decision rules are unspecified, raising concerns about label or repair-information leakage. Moreover, checking an LLM-proposed condition does not establish its sufficiency for sanitization, and agreement on a finite corpus does not prove whitelist-rewriter equivalence. The evaluation therefore neither isolates the verification layer’s contribution nor establishes that its taint-blocking decisions are sound for unknown vulnerabilities.

Questions for authors

What specific limitation of prior work does SanTA address, and what methodological contribution goes beyond combining LLM-generated hypotheses with taint analysis and SMT checks?

Which sanitizer patterns and vulnerability types do the 15 field findings cover, and what evidence supports generalization beyond these cases?

What inputs and rules do the fixed-version-specific gates use, does the verification layer improve results without them, and what guarantees establish sanitizer-condition sufficiency and whitelist-rewriter equivalence beyond finite testing?

Reviewer_JbNVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tackles taint analysis for Python programs. They implement a standard taint analysis, but use an LLM to identify when a sanitizer is actually active. The main problem is that a sanitizer is not universal - a lot of times it's context dependent. So they have an LLM generate a specification, and then discharge that to Z3 using other facts collected by their taint analysis. This outperforms the evaluated baselines.

Critical strengths

I think this is decent. It has a good neurosymbolic approach, and outperforms the baselines. Specifically, they're also able to find real-world bugs. Their ablations do generally show that it helps (better score than JUST using an LLM as a verdict).

Generally I think it feels like a pretty straightforward paper, in the sense that there is a well-defined problem, there are baselines, and their approach outperforms the baseline. Their ablations show that their infrastructure is necessary.

Critical weaknesses

The first weakness is one that I don't think needs to be addressed with experiments. But mainly, I'm not convinced that this approach would be better than just Mythos or Opus or Daybreak-Blue. But I think that's a bigger question for a lot of neurosymbolic stuff, and I don't think papers should be gated on access to those models. But I think it needs to be clearer whether this is a "we outperform the SOTA" or a "we allow smaller models to punch above their weight class". Because if it's the former, i think there should be more comparison with frontier models (sorry, I hate that standard too).

The next weakness is the writing. Section 6.3 (lines 422-431) is just ABSOLUTELY incomprehensible to me. That was the only part I really needed an LLM to help me understand. I still don't really know what it's saying to be honest, so I'm taking an LLM's word.

Questions for authors
  • could you explain what section 6.3 is saying and what the ablations are?
  • What is the benefit of this approach if more powerful models become more capable at verifying sanitizers?
1 comment

We sincerely appreciate the reviewer’s astute insights and constructive feedback, which prompted us to clarify the boundaries of our claims and substantially improve the rigor of our evaluation. Our responses to the specific points are provided below.

1. Response to the weaknesses

Thank you for your valuable comments, and we largely agree with your view. In the core contribution of SanTA, the LLM acts as a bridge connecting the taint analysis engine and the symbolic solver, so it only needs basic semantic understanding. Our goal is to let smaller models perform beyond their own weight class, which is why we do not need to compare with the most advanced frontier models. Nevertheless, we have added comparison experiments with frontier models such as DeepSeek-v4-flash and GLM-5.3-flash, whose data appear in the table below, and they demonstrate the advantage and significance of our method.

2. Could you explain what section 6.3 is saying and what the ablations are?

The ablation of Section 6.3 answers one question: does the F1 improvement of SanTA come from the adjudication layer of Section 4.3, or from the taint engine, the sanitizer vocabulary, or the prompts? Our ablation study runs two ablations on the expert-annotated subset of CVE-SAN-Python, varying only one factor at a time so that each comparison cleanly separates the adjudication from the rest of the system. The detailed content of Section 6.3 follows:

The two ablations of Figure 4 vary one factor at a time on the expert-annotated subset of CVE-SAN-Python, so each comparison separates the adjudication from the rest of the system. In Figure 4a, the engine, the sanitizer vocabulary, and the prompts are held fixed, and only the verdict issuer changes. With no condition declared, no rule can block the taint, so every unit is predicted ALIVE, recall is trivially 1.00, and F1 remains at 0.67. When the LLM issues the verdict directly, it discards alarms together with true vulnerabilities, and F1 falls to 0.62 with recall 0.80. Only when the symbolic layer verifies the declared conditions against the source code does F1 reach 0.73 with recall 1.00. Figure 4b decomposes the adjudication layer itself. It starts from the bare engine at F1 0.63 with recall 0.85 and adds the verification families of Section 4.3 one at a time. F1 rises monotonically to 0.69 with the transformer output-side verification, to 0.71 with the predicate-caller guard synthesis, and to 0.73 with the whitelist rewriter and its compile check, differential execution, and structural containment. Recall reaches 1.00 at the addition of the transformer output-side verification and remains at 1.00 for every subsequent configuration. The control run, which treats every sanitizer as an unconditional cleansing rule, collapses recall to 0.55 on the same units. The engine and the prompts are identical in every arm, and the movement in F1 is therefore attributable to the adjudication, whose monotone trajectory assigns a distinct share of the gain to each verification family.

We have rewritten Section 6.3 so that its logic reads clearly.

3. What is the benefit of this approach if more powerful models become more capable at verifying sanitizers?

In our method, the core vulnerability-detection task is decided by the taint analysis engine and the symbolic verifier. The LLM is not a core component of vulnerability discovery, it is only responsible for translating the engine's output into the assumption conditions that the symbolic verifier requires within the closed vocabulary, so our design leans toward using a smaller model to reach the expected effect. In Appendix B we have already measured, across different LLM backends, the F1 of condition-sanitizer identification and of end-to-end detection. The results show that the identification ability differs little across backends, and for the combined detection F1 the top is even Qwen3-next-80b-a3b-instruct, abbreviated as qwen3-80b, which means small models already meet the requirements of our method. For cost reasons we therefore no longer select the most advanced model.

Therefore, to assess the advantage of our method over more powerful models, we designed the following comparison experiment, to verify that a small model combined with our design performs better than larger models:

ArmModelPRF1
SanTAqwen3-80b0.5261.0000.690
SanTA(glm)glm-5.3-flash0.5151.0000.680
SanTA(dsv4)deepseek-v4-flash0.5051.0000.671
glmglm-5.3-flash0.5270.7800.629
dsv4deepseek-v4-flash0.5210.7920.628

Compared with the more capable large models glm-5.3-flash and deepseek-v4-flash, qwen3-80b combined with our design already matches the advanced DeepSeek-v4-flash and GLM-5.3-flash on precision, and attains the best F1. This demonstrates that small models can also reach the expected effect.

Reviewer_XEfhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a neuro-symbolic pipeline (SanTA) consisting of an LLM to propose condition hypotheses (conditions from a fixed vocabulary on types/vals) and a symbolic Solver (Z3) to verify them. The paper addresses the problem of vulnerability detection in Python due to dynamic typing, as here assumptions on sanitizer conditions are usually based on type assumptions, which might be violated during runtime. Further, it contributes the CVE-SAN-Python dataset, which consists of real-world vulnerabilities in Python packages

SanTA is evaluated on the proposed dataset and CVEfixes, reporting improvements in F1 scores on both.

Critical strengths

The idea of using both fuzzy LLMs together with symbolic verifiers for dynamically typed languages like Python is promising.

The results support the approach, and the addition of testing on real-world packages underscores the method's effectiveness on those packages. In addition, the baseline choice (symbolic and LLMs) seems fair.

Critical weaknesses

Writing and readability.

  • Semicolons (and colons) are used excessively, making some paragraphs and sentences hard to read throughout the paper. For example, 4.3 (a). This makes the paper sometimes read like a sequence of statements, with no transitions between sentences.
  • The RQs are never explicitly stated.

Small comments:  - In contributions: "a wild campaign" - phrasing

  • CVE abbreviation missing
  • Formatting:  Eq. equation X in multiple places
Questions for authors

Where do the original sanitizers and their conditions come from? What do the rules of the implemented system look like? How does the FactLayer generate types and possible values?

Sec. 4.2 How does the LLM generate hypotheses exactly? What is the prompt used, and what is its context?

What does the Input to Z3 look like? Can you illustrate these steps with the given running example?

3 comments

We are grateful for your careful reading and constructive suggestions, which have helped us clarify the scope of our claims and improve the rigor of the evaluation. We respond to the issues raised in the "Weaknesses" and "Questions" sections as follows.

1. Writing-related issues

We agree with your writing-related weaknesses and suggestions about our manuscript. We reviewed the use of semicolons and colons throughout the manuscript and removed the unnecessary ones. We rewrote the passages with more logical language and added explicit statements of the research questions. The complete modifications appear in the revised manuscript.

Then we added an explicit statement of each research question to the heading of its subsection in the results section.

RQ1: How Effective Is SanTA Compared with Existing Methods?

RQ2: How Effective Is Conditional-Sanitizer Identification?

RQ3: Which Component of SanTA Contributes to Its Performance?

RQ4: Can SanTA Discover New Vulnerabilities in Real Python Libraries?

Inappropriate wording in the Contributions section has been fixed, we have updated the abstract to use the same terminology throughout. The full term of CVE has been spelled out, and the formula issue has been corrected. For details, see the revised manuscript.

2. Where do the original sanitizers and their conditions come from? What do the rules of the implemented system look like? How does the FactLayer generate types and possible values?

The sanitizers and their effectiveness conditions are provided separately. The rules come from the built-in rule set that the taint engine already has, and they cover the common sanitizers in general Web, data processing, and other domains. The effectiveness conditions come from assumptions that the LLM declares inside the closed 7-category vocabulary under fixed constraints, and the FactLayer generates the types and values of the reachable inputs from the static information at the call site, serving as the yardstick for symbolic verification.

Since the common sanitizer logic in Python is generally the sanitizers in the standard library or generalizations derived from them, the vocabulary construction is based on this fact. The raw sanitizer conditions are assumptions that the LLM declares inside the closed 7-category vocabulary. The rule only provides the candidate, that there is a sanitization call here. What condition must hold for the call to actually sanitize is declared by the LLM after it reads the code along the chain.

The FactLayer derives the following quantities from static information per the rules below, and every result is recorded in the verdict record:

FieldExtraction ruleSource
Parameter static typeannotation > literal > isinstance narrowing > source-API default > call-chain back-propagationAST plus the engine's source pattern table, e.g., request.args.get maps to str
Parameter value seta literal argument maps to const, a constant assignment inside the caller function maps to a value set, otherwise unconstrainedcall-site AST plus constant propagation inside the caller function
Type-source annotationeach type carries torigin, ranging over annotation, literal, isinstance_narrow, source_api, call_chainvisible in the verdict record

3. Sec. 4.2 How does the LLM generate hypotheses exactly? What is the prompt used, and what is its context?

The LLM's hypothesis generation is implemented through prompt engineering. Every call involves slice identification, condition hypothesis, and rewriter compilation. Taking the main pipeline as an example, the prompt consists of the following four parts.

  1. Task setting. "You are auditing ONE concrete taint path in a Python project. Attacker-controlled data enters at a source call and flows through the FULL call chain shown below … toward a security sink."
  2. Context. The set of functions expanded by the chain expansion module, where each function is given its file and function location together with its depth, followed by its body. The context budget is a depth of at most 4, at most 18 functions, at least 2 slots per depth, and at most 1200 characters per function.
  3. Output schema.
    • Part 1 identifies the core sanitizer by reading the code rather than the function name.
    • Part 2 names the data parameter, i.e., the formal parameter that carries the attack data.
    • Part 3 declares the effectiveness conditions inside the closed 7-category vocabulary, where each category has a fixed JSON shape and is accompanied by conciseness rules, i.e., no character-by-character enumeration, subs of at most 6, and a "starts with X" pattern must be written as a single negated regex ^X.
  4. One self-correction round. When the parse fails or the core function is not on the chain, a mechanical error message is fed back and the model answers one more round.

4. What does the Input to Z3 look like? Can you illustrate these steps with the given running example?

The input to Z3 consists of three parts, the fact class FF, the condition φi\varphi_i, which the trusted encoder maps over the full 7-category vocabulary into Z3 expressions, and the guard-correspondence check. For every condition we ask two questions, i.e., we check the satisfiability of F∧φF\land\varphi and of F∧¬φF\land\neg\varphi, and we judge the condition violated, established, or attacker-violable according to sat or unsat. For a violated condition we then add the guard-existence check, and only when a guard GG exists such that F∧¬φ∧¬GF\land\neg\varphi\land\neg G is unsatisfiable do we let the flow through. Below we demonstrate this end-to-end with CVE-2025-61911:

Step 1, slicing and identification, for the vulnerable pole

  • The engine candidate hits the escape_filter_chars function of python-ldap, through the dual channel of the sanitizer rules and the vocabulary.

  • After chain expansion the LLM sees the entry escape_filter_chars together with 3 functions on the chain, i.e., chain_size is 3.

  • The LLM's output follows, and the raw field can be checked verbatim:

    {
      "core_sanitizer": "escape_filter_chars",
      "data_param": "assertion_value",
      "conditions": [
        {"param": "assertion_value", "kind": "type_constraint", "type": "str"},
        {"param": "assertion_value", "kind": "regex_constraint",
         "pattern": "^[\\x00-\\x7f]*$", "mode": "match", "negated": false}
      ]
    }
    

Step 2 — Z3 input, output, and intermediate state

vulnerable pole, verdict ALIVE, flow real, belongs to TP.

ObjectZ3 input, i.e., the real assertion or encodingZ3 outputIntermediate state and conclusion
Fact class FOr(1 == v_escape_mode), where the fact type of assertion_value is list mapped to sort other, ty is the constant 4 and has no value variable, and the literal argument 1 of escape_mode is encoded as const[1]——
Condition 1, i.e., type is strexpr1 is z3.BoolVal(ty_av == 0), and the result is False, i.e., BoolVal(4==0)the solver first asserts Or(1 == v_escape_mode), then asserts False, and check returns unsatthe first question is unsat and the condition is judged violated, so status is violated in the frozen record. The second question short-circuits and is not executed.
Condition 2, i.e., regex ^[\x00-\x7f]*$the parameter sort is other and there is no string value variable, cond_to_z3 returns None, and no solver call is made—status is unencodable and resolution is unknown, with the fail-closed strategy adopted.
Guardguard_predicates returns the empty set for the vulnerable core, i.e., there is no encodable rejection branch—the violated condition has no guard to rescue it, the flow is real, and the unit is judged ALIVE, i.e., TP.

fixed pole, verdict BLOCKED, flow blocked, belongs to TN. The only difference from the vulnerable pole is that the AST adds an entry guard.

ObjectZ3 input, i.e., the real assertion or encodingZ3 outputIntermediate state and conclusion
Fact class Fthe same as the vulnerable pole, the fact class at the call site is unchanged and is still a non-str class—condition 1 is still violated
Guard Gthe new if not isinstance(av, str): raise statement is encoded as G = z3.Not(z3.BoolVal(ty_av == 0)), i.e., Not(False), and the guard trigger expression is True—recognized as a fail-closed guard on assertion_value, and caller_guards lie in the same ring as the in-core guard.
Question 3, the guard-correspondence checkjudge F ∧ ¬φ₁ ∧ ¬G, executing assert Or(1 == v_escape_mode), assert Not(False), and assert Not(Not(False)) in turncheck returns unsatevery value that violates type is str necessarily triggers the entry check and is rejected, status is violated+guard_matched, and resolution is effective.
Condition 2the same as the vulnerable pole, still unencodable—by the vacuous-sibling rule, this input class is already rejected as a whole by the guard of condition 1, so it is removed from the conjunction, status is unencodable|vacuous_sibling, and resolution is effective.
Conclusion——all conditions are effective, and a decisive data condition exists, i.e., a violation rescued by a guard, so the flow is blocked, and the unit is judged BLOCKED, i.e., TN.

The same LLM hypothesis, the same Z3 encoding, and the same guard extraction derive real and blocked on the two poles respectively. Every state in the verdict record, including violated, guard_matched, unencodable, vacuous_sibling, effective, real, and blocked, is verbatim visible and auditable.

Thanks for your reply. My points regarding the presentation were addressed by the authors and incorporated into the manuscript. Further, technical questions were also satisfactorily addressed. I am therefore increasing my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.