acceptodds
Under review as a conference paper at ICLR 2027

LLM-DE: Jailbreaking LLM-based Malware Detection via Prompt Injection and Safety Alignment Exploitation

Abstract

Malware detection is entering a third generation in which Large Language Models (LLMs) reason over textual renderings of program features, promising contextual understanding that classical byte- and feature-space classifiers lack. This shift relocates the attack surface: the verdict is computed over text derived from the file, a channel that is fully attacker-controllable. We present LLM-DE, a systematic study of adversarial evasion against LLM-based malware detectors. LLM-DE formalizes four attack categories—prompt injection via feature descriptions (PIPR-A), safety-alignment exploitation (SAE-A), parsing-error induction (PEI-A), and semantic camouflage (SC-A)—each targeting a distinct stage of the LLM detection pipeline under a strict functionality-preserving, feature-addition-only constraint. We evaluate end-to-end against detectors built on six backbone models spanning three orders of magnitude. The evaluation uncovers a capability prerequisite: sub-threshold backbones issue input-invariant verdicts, so the detector—not the attack—fails first. On the smallest discriminating backbone (Phi-3-mini, measured), every payload-carrying construction flips every measured malicious sample to confidently benign (residual malicious probability for the unified optimized attack), and effectiveness varies across backbones in a per-channel manner—injection is near-universal, while alignment exploitation, semantic camouflage, and truncation scale with alignment strength, semantic priors, and context budget, respectively. A refusal-gate mechanism analysis—grounded in how aligned LLMs attribute harmful intent rather than harmful keywords—explains the naive variants' failures and yields optimized constructions whose measured outcomes exceed their disclosed projections. Against four baseline attack strategies, single-channel baselines (payload-only, suffix-only, or feature-space-only) evade at most one detector family, while the joint LLM-DE construction evades both the LLM detector and a trained classical DREBIN-SVM (99.7% ESR) simultaneously. Our results indicate that moving malware detection to LLMs does not remove adversarial vulnerability; it re-opens it along four channels that no single backbone property closes.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large languadge model;adversarial attack;prompt injection;detector evasion;malware detection

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6hrGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies adversarial evasion against malware detectors that use large language models (LLMs) to reason over textual renderings of program features. It argues that converting attacker-influenced APK features into natural language creates attack channels beyond conventional byte- or feature-space manipulation. The paper introduces LLM-DE, a framework organized around four attack categories: prompt injection through feature descriptions (PIPR-A), exploitation of safety-alignment and refusal behavior (SAE-A), parsing-error and truncation induction (PEI-A), and semantic camouflage using benign application signatures (SC-A). It further proposes mechanism-driven optimized variants, including safety inversion, two-sided semantic neutralization, targeted truncation, and a unified composition, together with informal analyses of submodular feature selection, stealth, and asymmetric transfer to classical detectors.

Validation uses a synthetic 215-dimensional DREBIN-style Android feature corpus, a fixed feature-to-text converter and system prompt, six instruction-tuned LLM backbones, and two classical detectors. The measured end-to-end LLM results are primarily obtained on Phi-3-mini using only three malicious samples; Qwen2.5-0.5B/1.5B are shown to produce input-invariant outputs, while results for Qwen2.5-7B, Llama-3.1-8B, and a GPT-4-class model are pilot-anchored projections rather than measurements. On Phi-3-mini, PIPR-A, SAE-A+, SC-A+, and Unified+ flip all three measured malicious samples, whereas PEI-A+ does not. Feature additions also transfer strongly to a linear DREBIN-SVM, with up to 99.7% ESR for PIPR-A and 79.1% for Unified+. The paper concludes that LLM-based malware detection remains vulnerable to both text-level and feature-level adversarial manipulation.

Critical strengths
  1. The paper identifies a timely and underexplored security problem. Rather than treating an LLM detector as an ordinary classifier, it analyzes the full feature-to-text-to-model pipeline and makes explicit how attacker-controlled representations can become instruction channels. The four-way taxonomy, covering prompt injection, alignment exploitation, parsing/truncation, and semantic camouflage, provides a useful vocabulary for auditing LLM-based security systems beyond malware detection. This framing is consequential because it connects malware evasion with broader prompt-injection and agent-security concerns, and it suggests concrete defensive requirements such as separating trusted instructions from untrusted feature text. Although the current validation is insufficient for strong acceptance, the problem formulation and taxonomy could motivate substantial follow-up work.

  2. The mechanism-driven analysis produces informative, testable hypotheses rather than reporting only attack success rates. The paper explicitly investigates why naive SAE-A, PEI-A, and SC-A fail on Phi-3-mini, then proposes targeted variants based on refusal intent, two-sided evidence control, and deterministic truncation. The resulting measured changes, including 0/3 to 3/3 flips for SAE-A+ and SC-A+, provide a useful proof-of-concept that analyzing model behavior can guide adversarial construction. The study also reports negative results, per-sample probabilities, baseline comparisons, and transfer asymmetry between a linear SVM and an MLP. These details improve interpretability and reproducibility, even though they do not overcome the paper’s limited sample size and synthetic evaluation setting.

Critical weaknesses
  1. The central empirical claims rest on an extremely small and narrow evaluation. The decisive end-to-end results are obtained with one discriminating backbone, Phi-3-mini, and only three malicious samples. The other small models are input-invariant and therefore do not constitute meaningful detector evaluations. A three-sample sweep cannot establish robustness, reliability, or generalization across malware families, feature combinations, prompts, or random seeds; a 3/3 success rate can easily reflect sample selection or a pipeline-specific artifact. The paper also uses a synthetic feature corpus rather than executable, repackaged APKs, so it does not demonstrate that the proposed additions are realizable while preserving malicious behavior. Because the paper’s strongest conclusions concern systematic vulnerability and broad attack-surface coverage, this limited evidence is consequential and insufficient for an ICLR-level empirical security claim.

  2. The headline cross-backbone results are projections rather than measurements. The table that supports claims about Qwen2.5-7B, Llama-3.1-8B, and GPT-4-class detectors is generated by a pilot-anchored model using assumed capability and channel-susceptibility parameters. These estimates are explicitly not experimental outcomes, yet they are presented alongside measured ESR values and are repeatedly used to argue that attacks strengthen with model scale and alignment. Published jailbreak priors do not validate this particular malware-detection pipeline, and the model’s independence assumptions are untested. The measured Phi-3-mini outcomes even contradict some projected behavior for the naive attacks. Consequently, the paper does not currently establish transfer across backbones or the claimed near-universal effectiveness; it establishes only a promising hypothesis that requires direct experiments on the advertised target models.

  3. The threat model does not convincingly establish an executable payload path. The framework assumes that adding benign permissions, APIs, or intents can cause the feature-to-text converter to emit arbitrary instruction payloads, but the converter is otherwise described as deterministic and based on canonical feature descriptions. If the payload instead comes from app labels, library descriptors, or string resources, the paper must specify which attacker-controlled fields are parsed, how they survive APK rebuilding, and whether the deployed detector actually includes them in its LLM input. Merely adding a feature does not generally let an attacker rewrite its canonical description. This unresolved mismatch affects the feasibility of PIPR-A and the unified attacks, and it also undermines the “functionality-preserving” claim because no real APK construction, installation, execution, or behavioral equivalence test is provided.

Questions for authors
  1. Clarification on measured versus projected evidence: Which numerical results in the main paper were obtained from direct end-to-end queries to each named backbone, and which were generated by the pilot-anchored projection model? In particular, can the authors confirm that no direct measurements support the Qwen2.5-7B, Llama-3.1-8B, and GPT-4-class ESR values in Table 1, and explain why these projections are used to support cross-backbone conclusions?

  2. Clarification on functionality preservation: Were any attack samples instantiated as valid APKs and checked for successful rebuilding, installation, execution, and preservation of the original malicious behavior? If not, what precisely does the paper mean by “functionality-preserving” beyond the feature-addition-only constraint, and which reported attacks rely on modifications that are not represented in an executable artifact?

  3. Clarification on the ESR success criterion: How many reported successes are actual MALICIOUS-to-BENIGN flips, versus refusals or unparseable outputs? Would the main conclusions, especially for SAE-A, remain unchanged if refusal and malformed responses were mapped to MALICIOUS under a fail-closed detector policy? Please also clarify whether the reported first-token verbalizer mass was calibrated or validated against the detector’s final classification decision.

1 comment
AuthorsOpenReview ↗

Direct-backend inventory, fail-closed ESR, and functionality-preservation correction

Thank you for the focused clarification questions.

Which results are direct? Direct end-to-end attack measurements exist only for Phi-3-mini on three malicious samples, with five benign clean controls. PIPR-A, SAE-A+, SC-A+, Unified, Unified+, B2, and B3 were directly queried in their respective batches; PEI-A+ did not flip the three samples. The released naive-Unified CSV contains only two rows although Table 15 reports three, so we will not use the missing third row as auditable evidence. The small Qwen2.5-0.5B/1.5B results are collapse/probe observations, not useful attack evaluations. We confirm that no direct measurements support the Qwen2.5-7B, Llama-3.1-8B, or GPT-4-class ESR values in Table 1. Those values, and the capability-scaled Phi-3 entries, come from the projection model. Classical SVM ESR, assumed channel susceptibility, and probe accuracy have no validated mapping to LLM ESR. We will remove the projected tables and stop using them for cross-backbone or capability conclusions.

What does functionality preservation mean here? No attack sample was instantiated as an APK, and no rebuilding, installation, execution, or malicious-behavior test was performed. In the submitted code it means only that a synthetic binary feature is not removed. All attacks operate on synthetic vectors; PIPR-A and unified variants additionally rely on a payload directly appended by our renderer. We have not demonstrated an attacker-controlled APK field that survives an unchanged extractor/converter into that position. Adding a feature cannot itself rewrite a canonical description. We will therefore remove “functionality-preserving” as an executable-artifact claim and narrow the threat model to the constructed feature-to-text pipeline.

How are successes counted? The original CSVs contain only a top-20 first-token score, not final generated responses. The scorer aggregated B*/M* token prefixes and renormalized over their sum; this was not calibrated against complete generations, was not a complete class-label likelihood, and returned 0.5/0.5 when class tokens were absent. We therefore reran all 26 original cases and saved final generations. Exact whole-class token mass over the full first-token vocabulary was zero in all 26 cases because both labels span multiple tokens. Final generations produced no REFUSAL or semantically unparseable outputs, but two lacked the required prefix. For the three clean-detected malicious samples, explicit BENIGN counts were: clean 0/3; PIPR-A 3/3; B2 3/3; SAE-A+ 3/3; SC-A+ 3/3; PEI-A+ 0/3; Unified+ 3/3. Two outputs used bare first-line BENIGN instead of CLASSIFICATION: BENIGN. Under strict-format fail-closed scoring they are malformed and blocked, so PIPR-A and Unified+ become 2/3; other counts are unchanged. Thus semantic fail-closed counts are unchanged, while strict formatting reduces two attacks. Complete-label sequence scoring on nine clean cases plus one PIPR-A case disagreed with generation for 2/5 benign controls, so this limited audit is not a calibration validation. All raw responses are preserved.

For fail-closed ESR, we condition on the three originally detected malicious samples and count only an explicit generated BENIGN as an escape. MALICIOUS, REFUSAL, and OTHER all remain blocked. This is the appropriate policy when uncertain output is quarantined rather than admitted. The new diagnostic audits measurement on the same samples; it does not remedy the tiny sample, establish calibration on a representative dataset, or demonstrate APK realizability.

We agree that the supported conclusion is only a limited proof of concept on this pipeline. Broad claims of systematic vulnerability, cross-backbone transfer, and practical malware evasion require direct large-scale backbone tests and executable APK/system-path validation.

Reviewer_noCROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how to evade malware detectors that classify textual descriptions of Android application features. It proposes four attack categories: prompt injection, safety-alignment exploitation, context truncation, and semantic camouflage. The attacks combine two things: adding features to the app’s binary feature vector and inserting extra text payloads. The paper also claims three theoretical results: a guarantee for greedy feature selection, a stealth guarantee against human analysts, and asymmetric transfer between LLM-based and classical detectors.

Experiments use a synthetic corpus with 215 features. The main measured evaluation of attacks on an LLM detector uses three malicious examples on Phi-3-mini. Prompt injection and several other variants that carry a text payload flip all three classifications. The naive alignment, camouflage, and overflow variants do not. Two smaller Qwen models give the same classification regardless of input under the tested protocol. Attack rates for larger models are projections, and the paper says so explicitly. Additional experiments on synthetic feature vectors evaluate an SVM and an MLP.

Critical strengths
  1. A relevant attack surface, with a proof of concept that can be inspected. Malware-analysis systems that read text an attacker can influence deserve scrutiny for this kind of failure, where the model follows injected instructions. Tables 11–12 usefully separate attacks that change only the text payload from attacks that change only the features. Payload-only changes flip the tested LLM classifications, whereas feature-only changes affect the classical detector. The paper also reports the variants that failed and labels the larger-model results as estimates. These disclosures make the limited pilot evidence easier to assess.
Critical weaknesses
  1. The measurements do not support the claimed breadth or practical effectiveness. The abstract and conclusions make claims about model capability, alignment mechanisms, effectiveness across models, and realistic malware evasion that go well beyond what was measured. Measured evidence. Appendix D.3 states that the measured attack campaign stopped after three malicious samples. Even before any attack, the same detector correctly classifies only two of five benign controls (Table 15). This evidence cannot establish that the detector reliably detects malware, that evasion is near-universal, or that there is a general capability threshold. Moreover, the payload-only baseline already succeeds on all three samples. The measured LLM results therefore show no advantage for the combined attack over plain prompt injection. Projections. The results for larger models cannot fill this gap. Appendix D.4 computes projected LLM evasion rates by combining an assumed susceptibility of each attack channel with the measured evasion of the classical detector. The paper establishes no validated relationship between SVM evasion and LLM evasion. It also establishes none between probe accuracy and susceptibility to the proposed channels. For PIPR-A, the projections of about 99.8% come largely from the SVM’s 99.7% evasion rate. The trends across models therefore reflect the assumptions of the projection model, not observations. Labeling these numbers as estimates is appropriate, but that does not make them experimental evidence or justified lower bounds. Realizability. All samples are synthetic feature vectors, and repackaging real APKs is left to future work. Adding binary features does not by itself show that a real app can be transformed validly while keeping its functionality. Nor does it show that the attacker’s payload survives extraction into the detector’s input. Pierazzi et al. (IEEE S&P 2020) draw exactly this distinction between changes in feature space and attacks that can actually be carried out on real programs. The end-to-end result here therefore concerns the constructed text-classification pipeline, not a deployment that goes from APK to verdict.
  2. The optimization and stealth guarantees are unsupported, and Proposition 4 has a simple counterexample. Section 5.1 and Appendix A argue that evasion probability is submodular, based on feature dilution and threshold crossing. But applying a threshold to a monotone score does not in general preserve diminishing returns. Consider a deterministic classifier that is evaded only once at least two features have been added: g(S) = 1 if |S| ≥ 2, and 0 otherwise. For distinct features a and b, adding b to the empty set gives a gain of g({b}) − g(∅) = 0. Adding b to {a} gives a gain of g({a, b}) − g({a}) = 1. This violates the diminishing-returns inequality in Proposition 4. Yet it satisfies the premise of the proof sketch, which is that dilution eventually crosses a classification threshold. As a result, the claimed (1 − 1/e) guarantee does not follow. The classical greedy result requires a monotone submodular objective (Nemhauser, Wolsey & Fisher, Mathematical Programming 1978). Optimizing an unspecified surrogate would also require a link between the surrogate’s quality and the true objective, and the paper provides none. Theorem 1 has a similar gap. The fact that the added features are common across apps places no stated constraint on the analyst’s anomaly function or on unusual combinations of features. The numerical frequency and suspicion thresholds therefore cannot establish that the modifications would go unnoticed by analysts. These problems are substantive because the paper presents the optimization guarantee and stealth as core contributions.
Questions for authors

My recommendation rests on three things: the small size of the measured evaluation, the reliance on projections, and the invalid approximation guarantee. Fixing them would take substantial new evidence and revised theoretical claims. The questions below are mainly meant to clarify what the current evidence supports. They concern the paper’s existing claims and do not ask for expansion to other attack domains.

  1. Measured evidence (clarification). Is there any measured LLM evaluation beyond the three malicious examples? Please separate conclusions supported by measurements from hypotheses generated by the projection model. What evidence justifies using the classical detector’s evasion rate as an input to projected LLM evasion?
  2. Attack realizability (clarification and demonstration). Which exact fields in the APK, controlled by the attacker, carry the payload? How does the fixed extractor/converter preserve them? Please distinguish modifying a string inside the application from modifying a canonical feature description. Can the claimed mapping be demonstrated on a real application without changing the trusted converter?
  3. Verdict measurement (clarification). How do you handle class labels that span multiple tokens, and labels missing from the top-20 log probabilities? Are probabilities normalized over the complete class labels? Please provide the generated responses or the full label likelihoods. Please also separate benign verdicts, refusals, and malformed outputs. A very small probability on the malicious token does not by itself show a confident benign classification. A refusal counts as evasion only if the downstream system explicitly lets inputs through when the model refuses.
  4. Theoretical claims (clarification). What additional assumptions rule out the threshold counterexample to Proposition 4? What assumptions connect feature frequencies to analysts’ anomaly scores? What conditions on the surrogate support the claimed approximation guarantee? If these cannot be established, the corresponding results should be presented as heuristics or hypotheses.
1 comment
AuthorsOpenReview ↗

Accept the counterexample; narrow empirical claims and correct verdict measurement

Thank you for the precise critique. We agree with the counterexample and with the distinction between measurements, hypotheses, and realizable attacks.

Measured evidence and projections. There is no measured LLM attack evaluation beyond the three Phi-3-mini malicious examples; the five benign controls are clean controls, and only 2/5 were classified correctly. The small Qwen results are input-invariant probes, not meaningful detector evaluations. No direct ESR supports Qwen2.5-7B, Llama-3.1-8B, or GPT-4-class. There is also no validated justification for using classical-detector ESR as an LLM-evasion component, or probe accuracy as channel susceptibility. We will remove the projected ESR tables and all cross-backbone, capability-threshold, and near-universal claims. B2 payload-only, PIPR-A, and Unified+ each flipped 3/3, so we withdraw any measured LLM-side advantage for the combined attack over plain injection.

Realizability. No real APK was transformed or executed. Our renderer directly appends an attacker string after canonical feature text. We have not identified and validated an APK field that an unchanged extractor routes into this payload slot. Modifying an app label/string resource is therefore not equivalent to the demonstrated modification of a canonical feature description. “Feature-addition-only” describes only the synthetic vector operation; it does not establish rebuilding, installation, execution, or malicious-behavior preservation. We will restrict the result to the constructed text-classification pipeline.

Verdict measurement. The original implementation used top-20 first-token logprobs, aggregated tokens by B*/M* prefix, and normalized only their sum; if neither prefix appeared, it returned 0.5/0.5. Thus it was neither a full-vocabulary class mass nor a complete multi-token label likelihood, and the old CSVs did not save final generations. We reran the same 26 cases and saved every generated response. Exact whole-class single-token coverage was 0/26: Phi-3 tokenizes the labels as multiple tokens, confirming the concern. For the three originally detected malicious samples, generated BENIGN counts were: PIPR-A 3/3, B2 3/3, SAE-A+ 3/3, SC-A+ 3/3, Unified+ 3/3, PEI-A+ 0/3; clean was 0/3. There were no refusals. Two successes used a bare first-line BENIGN rather than the requested prefix. They count under semantic fail-closed scoring, but under strict-format fail-closed scoring they are malformed and blocked, reducing PIPR-A and Unified+ to 2/3; the other counts are unchanged. Complete-label likelihoods for nine clean cases and one PIPR-A case exposed disagreements with generation on 2/5 benign controls, so we do not claim general calibration. Raw responses and per-row scores are available for inspection. Under fail-closed evaluation, only explicit BENIGN counts as escape; MALICIOUS, REFUSAL, and OTHER are blocked. This diagnostic does not increase the sample size.

Theory. Your function g(S)=1[∣S∣≥2]g(S)=\mathbf{1}[|S|\geq2] directly violates diminishing returns while retaining threshold crossing. Monotonicity and thresholding do not imply submodularity. We withdraw Proposition 4 and its (1−1/e)(1-1/e) claim. For the linear surrogate, score change is additive, so selecting the kk most negative feasible weights is exact for that surrogate only; there is no established surrogate-to-LLM approximation bound. We will call the method a surrogate-based heuristic.

Theorem 1 is also withdrawn: marginal feature frequencies place no constraint on an analyst's anomaly function or unusual feature combinations, and the numerical thresholds do not imply perception safety. Theorem 2 will be retained only as a conditional linear-score explanation under explicit weight assumptions, not a general transfer theorem. We agree that these changes narrow both the theoretical and empirical contributions.

Reviewer_TpqMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper performs a study of LLM jailbreaking in the malware detection space. The authors introduce four different attack channels which characterise the prompt injection attack surface. With a synthetic DREBIN style dataset initial results suggest that attacks combining feature-space perturbations with adversarial textual payloads can evade the evaluated LLM detectors.

Critical strengths
  1. With the proliferation of LLMs in many workflows this paper highlights an interesting angle that attacker controlled malware can effect both the initial feature extraction, but then the NLP representation that a LLM would see. Given the growing interest in LLM aided security analysis this is a relevant concern.

  2. The proposed four channel taxonomy is interesting, and can go beyond this work to more general thinking about robustness of LLM malware detectors. In general, the taxonomy is a nice step to try and consider in this domain more granularity than a single monolitic prompt injection surface.

Critical weaknesses
  1. The paper is essentially an empirical one, so it is a weakness that the cross-backbone conclusions heavily utilise pilot-anchored projections rather than regular end-to-end measurements which would be expected from an empirical paper at a top venue. The author's are upfront about it, and clearly define their estimation process in the appendix, the strongest empirical evidence is only on the Phi-3-mini model. Thus, many of the conclusions feel unsupported without the rigour of directly measures performances.

  2. Perhaps I am missing something, but the paper describes a dataset of 2,000 training (balanced) and 600 test applications (Section 6.1). But then in Table 1, and in the discussion sections in 6.2 and onwards only three samples were used? E.g. batch n=3, 0/3 used in the discussions. If this is the case I really don't see a path for acceptance for an empirical paper basing its main premise over 3 samples.

  3. The synthetic and simplified nature of the data weaken the security claims. The attacks are not actually run through APK transformations, and end-to-end repackaging/functionality preservation is left to future work. Hence, the paper runs attacks against a simulated feature-to-text detection pipeline vs actual realisation on malicious android apps.

  4. Several background works are cited (AppPoet and LAMD) which have several steps (multi-view prompt engineering, tier- wise code reasoning, downstream classifiers, etc). By comparison the detector analysed in this work is much simpler (feature-to-text converter and then an LLM decision). It's unclear then that the attacks studied in this work affect some of the motivating systems for the study - do the attacks function through the preprocessing, aggregation, and classification mechanisms discussed in the related works?

Questions for authors

See Weakness

1 comment
AuthorsOpenReview ↗

Agree on direct-measurement, realizability, and target-system limitations

Thank you for clearly separating the interesting attack surface from the empirical support. Your reading is correct: although the synthetic corpus contains 2,000 training and 600 test vectors, the main end-to-end LLM attack batch contains only three malicious samples, all on Phi-3-mini, plus five benign clean controls. The classical SVM/MLP tables use more synthetic vectors, but they are not additional LLM attack measurements. The benign detector itself classified only 2/5 controls correctly. We therefore cannot use the present batch to establish reliable malware detection, generalizable ESR, or cross-backbone behavior.

We also confirm that the Qwen2.5-7B, Llama-3.1-8B, and GPT-4-class entries are projections, not direct queries. The projection combines classical-side ESR with assumed channel susceptibility and capability parameters. There is no validated relationship that makes the former evidence for LLM evasion. Disclosure that these values are estimates does not make them suitable support for empirical conclusions. We will remove the projected result tables and the associated capability-scale and backbone-ordering claims. The small Qwen probe results show input-invariant collapse and will not be counted as detector evaluations.

The APK criticism is also correct. We did not rebuild, install, execute, or test malicious-behavior preservation for any attack. Our implementation changes synthetic binary features and directly appends text to the renderer. Consequently, “feature-addition-only” is only a vector-space constraint; it is not evidence of a functionality-preserving APK transformation. In particular, adding a permission/API/intent does not generally permit rewriting its canonical description, and we have not traced a concrete attacker-controlled APK string through an unchanged extractor into the LLM prompt. We will remove practical-APK and functionality-preservation claims.

Finally, we did not run AppPoet or LAMD. Our target is a deliberately simple deterministic feature-to-text converter with first-token verbalizer readout. Those systems' multi-view preprocessing, reasoning aggregation, and downstream classifiers may block, transform, or discard the assumed carrier. The present results therefore cannot be extrapolated to them. We will narrow the claims to this constructed pipeline and present the taxonomy as a set of testable hypotheses.

A credible empirical revision requires a substantially larger direct multi-backbone evaluation and an executable APK-to-prompt trace with rebuild/install/run/behavior checks, followed by tests through the actual preprocessing and aggregation paths of motivating systems. We agree that these are necessary evidence, not optional exposition changes.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Corrections to empirical scope, verdict measurement, and theoretical claims

We thank all three reviewers for identifying substantive gaps. After auditing the manuscript, code, and raw logs, we agree that several central claims exceeded the evidence. We will make the following corrections rather than defend those claims.

Measured versus projected evidence. Direct end-to-end attack queries were run only on Phi-3-mini, for three malicious samples, with five benign clean controls. Only 2/5 benign controls were classified correctly. The two small Qwen models showed input-invariant probe behavior and are not meaningful detector validations. No direct ESR measurements support Qwen2.5-7B, Llama-3.1-8B, or GPT-4-class. We will remove the projected ESR tables and all capability-threshold, near-universal-effectiveness, and cross-backbone ordering claims. Classical SVM ESR and assumed channel susceptibilities are not validated proxies for LLM ESR.

Comparative claim. B2 payload-only injection, PIPR-A, and Unified+ each flipped 3/3 measured malicious samples. Thus the current LLM-side results show no advantage of the combined construction over plain injection; we will withdraw that interpretation.

Realizability and target systems. No attack was instantiated, rebuilt, installed, executed, or behaviorally checked as an APK. “Feature-addition-only” applies only to our synthetic vectors, not to demonstrated functionality preservation. The renderer directly appends the payload; we have not established an attacker-controlled APK field that survives a fixed extractor/converter into that slot. We also did not evaluate AppPoet or LAMD. We will limit the study to the constructed feature-to-text verbalizer pipeline and remove claims about deployed systems.

Theory. Reviewer noCR's threshold example is a valid counterexample to Proposition 4. We withdraw submodularity and the (1−1/e)(1-1/e) guarantee. Greedy selection remains only a linear-surrogate heuristic: for an additive surrogate score, top-kk weight selection optimizes that surrogate, with no approximation guarantee for true LLM evasion. Theorem 1 is also unsupported because feature frequency does not constrain an analyst's anomaly function or feature combinations; we withdraw the general stealth guarantee. Theorem 2 will be reduced to a conditional linear-surrogate explanation, not a general asymmetric-transfer theorem.

Verdict audit. The original scorer aggregated B*/M* prefixes among top-20 first-token logprobs and renormalized only the collected mass; it did not store final generations. Our same-sample audit found zero exact whole-class single-token coverage in all 26 cases because the labels span multiple tokens. Saved generations contained no refusals; 24/26 followed the requested prefix, while two used bare first-line BENIGN. Under semantic fail-closed scoring, PIPR-A, B2, SAE-A+, SC-A+, and Unified+ remain 3/3 explicit-BENIGN escapes, PEI-A+ remains 0/3, and clean remains 0/3. If the required CLASSIFICATION: prefix is enforced, the two bare outputs are malformed and blocked, reducing PIPR-A and Unified+ to 2/3. Complete-label likelihoods disagreed with generation on 2/5 benign controls, so we do not claim general calibration. Raw responses and per-row results accompany the reviewer-specific replies. This is a measurement audit, not an expanded empirical evaluation.

These corrections substantially narrow the paper. We agree that direct multi-backbone evaluation, a larger representative sample, and executable APK/system-path validation are necessary before broad security-effectiveness claims can be supported.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.