acceptodds
Under review as a conference paper at ICLR 2027

GibberCraft: ADMM-Based System-Prompt Obfuscation for LLM Applications

Abstract

System prompts define the tasks, response contracts, and workflow rules of large-language-model applications. Their readable text is consequently a valuable implementation artifact, yet prompt extraction and reverse engineering can recover secret prompts from deployed services. We study whether we can replace a readable system prompt with a gibberish-like sequence of ordinary tokens that preserves the original behavior. This task is challenging because the replacement prompt must be optimized in a discrete token space. We formulate this task as constrained behavior-preserving optimization and leverage the alternating direction method of multipliers (ADMM) to separate continuous behavioral optimization from the hard vocabulary constraint. We further introduce Gumbel-perturbed hard-token sampling to account for hard-projection effects during optimization, improving optimization robustness. We evaluate behavioral fidelity, success rates, and protection against prompt injection across multiple tasks and model families. We achieve a 100% success rate on all tasks and match or outperform prior work, and achieve up to 9.3 speedup. These results demonstrate the feasibility of replacing readable system prompts with gibberish-like hard-token prompts for prompt protection.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM, ADMM, prompt obfuscation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2ejzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GibberCraft, a method for replacing readable system prompts with obfuscated hard-token sequences while preserving the original instruction-following behavior. It formulates prompt optimization using ADMM to alternate between continuous embedding optimization and discrete token projection, augmented with Gumbel-perturbed hard-token sampling to mitigate projection-induced degradation. Experiments on Llama 3.1 8B and Qwen3 8B report improved behavioral fidelity over Pape et al. (2025), 100% task success on two controlled instruction-following tasks, and up to 9.3× optimization speedup.

Critical strengths
  1. Clear formulation and promising empirical improvements. The paper provides a clear ADMM-based formulation of discrete prompt obfuscation, combining continuous optimization with hard-token projection and Gumbel-perturbed sampling. The experiments demonstrate promising improvements in instruction-following success and computational efficiency over the closest prior work.
Critical weaknesses
  1. Limited evaluation of behavioral fidelity and confidentiality. The instruction-following evaluation primarily measures output format rather than semantic correctness or completeness, while the security evaluation focuses on lexical prompt extraction rather than recovery of the underlying instructions or business logic. These evaluations do not fully substantiate the paper's claims of behavioral preservation and prompt protection.

  2. Limited task complexity and generalizability. The experiments focus on relatively simple tasks with short system prompts (10 to 32 tokens). It remains unclear whether the method generalizes to realistic applications involving longer prompts, multiple constraints, and complex workflows.

Questions for authors

See weaknesses, and:

  1. The bold font in Table 1 does not consistently indicate the best performance, potentially misleading readers. The authors should clarify the formatting and qualify their claims, particularly where baseline outperforms GibberCraft.

  2. The reported 100% instruction-following success rates are potentially misleading, as the LLM judge primarily assesses output format. Could the authors include task-specific correctness metrics, such as factual consistency for headline generation and entity-level F1 for name extraction?

  3. Could the authors compare against other continuous-to-discrete prompt optimization methods beyond GCG?

  4. Why does GibberCraft use only 128 training examples compared with 640 for Pape et al., how are these subsets selected, and how sensitive are the results to training-set size?

1 comment

Thanks for your constructive feedback and suggestions. We clarify the concerns as follows:

  1. Font Format and Advantage Clarification: We have removed the method-based bolding and clarified the comparison. GibberCraft achieves better similarity scores in most reported comparisons. Our contribution is efficient preservation of task behavior, with the clearest advantages appearing in tasks that require more specific instruction following.

    On the simpler QA and summarization tasks, both methods produce responses that resemble those generated with the readable prompt, and several similarity scores are close. These metrics capture different aspects of similarity, including lexical overlap and embedding-based similarity. For example, on Qwen summarization, the method of Pape et al. achieves higher BLEU, while both methods obtain cosine similarity of 0.94, and GibberCraft achieves higher BERTScore. A marginal advantage on an individual metric does not establish overall superiority in this case.

    In addition, QA and summarization overlap substantially with the models’ default capabilities, as indicated by the nontrivial response similarity obtained with an empty system prompt. A higher similarity score is therefore informative, but does not by itself establish better preservation of the requested instruction. This also motivates our evaluation on headline generation and name extraction tasks.

    We therefore additionally evaluate headline generation and name extraction, where the empty-prompt model does not naturally produce the required output behavior. GibberCraft achieves substantially stronger results overall in these controlled settings. Most notably, it achieves 100% task-form compliance on name extraction for both models, compared with 3% and 9% for Pape. These results demonstrate our clearest advantage in preserving specific instruction-following behaviors.

  2. Comparison with continuous-to-discrete prompt optimization: Following your suggestion, we added PEZ, a continuous-to-discrete prompt optimization method, as a baseline. Its results are reported in Tables 1 and 3. GibberCraft achieves stronger response-similarity results overall across the evaluated settings.

  3. Entity-Level F1: We've included entity-level F1 for name extraction. Results are shown in Table 14 in the appendix.

PromptLlama 3.1 8BQwen3 8B
Readable baseline91.6193.19
Obfuscated prompt89.8177.13
  1. Training set size: We apologize for the ambiguity in the original description. The 640-example setting refers to Pape et al.'s original work, not our baseline reproduction. In our QA and summarization experiments, both GibberCraft and the Pape baseline used 128 training examples. We chose this size for practical computational reasons: GibberCraft requires approximately two hours per run in these settings, and Pape et al.'s work takes roughly 2–3 hours. Training and evaluation examples were randomly sampled from the dataset. We have clarified these settings in the manuscript.
Reviewer_sqmgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates "system prompt obfuscation": without modifying the application, API, or inference interface, a human-readable system prompt PP is compiled into a sequence of "gibberish-like" hard tokens GG. This transformation ensures that the model’s behavior—across a distribution of user queries—remains as consistent as possible with that of PP, while simultaneously reducing the risk of directly exposing readable instructions or underlying business logic.

Key contributions include: a system prompt obfuscation method based on the Alternating Direction Method of Multipliers (ADMM) for hard-token optimization; the use of Gumbel hard-token sampling to enhance robustness; and preliminary validation across various models and tasks, with reported speedups of up to 9.3×.

Critical strengths

The methodological contribution is clear and transferable: ADMM is employed to decompose discrete vocabulary constraints into soft optimization and hard projection steps, while Gumbel-based hard-token sampling is used to bridge the gap between training and deployment. This approach is more principled than position-by-position search (as seen in GCG), explaining why it enables the joint optimization of all positions and reduces the computational overhead associated with long prompts.

The approach offers practical value for real-world deployment: the obfuscated prompts remain standard sequences of hard tokens, allowing them to serve as direct drop-in replacements for system prompts without requiring any modifications to the application or inference stack. Preliminary results—spanning Llama and Qwen models across four distinct task categories—demonstrate the feasibility of achieving "behavioral preservation" while rendering the prompts unreadable.

Critical weaknesses

The safety evaluation is insufficient to support the core claim of "protecting system prompts." The experiments cover only predefined prompt-extraction queries; they fail to assess adaptive attacks, black-box behavioral cloning, functional/logical inference, or the use of another LLM to interpret or reverse-engineer "gibberish" prompts. Furthermore, the ethical statement acknowledges that the method cannot prevent high-level functional inference. The "target" in Table 4 is ambiguous: if the "exact/approx" figures for "decoded hard-token prompts" represent the proportion of deployed hard-token sequences recovered by an attacker, they do not directly demonstrate the level of protection afforded to readable prompts. Conversely, if the target refers to the readable prompts themselves, then the ~29/105 "exact" recovery rate observed on Llama significantly undermines the claims regarding protection.

The assessments of behavioral preservation and computational efficiency lack credibility. In Table 1, the instruction-following evaluation is limited to only two tasks and a single random seed. The speed comparison fails to specify whether the benchmarks were conducted using identical hardware, prompt lengths, token budgets, or implementations consistent with the work of Pape et al.; consequently, the reported 7.76–9.33× speedup is questionable. The analysis also lacks sensitivity studies for various parameters, ADMM convergence curves, and an evaluation of variance across multiple random seeds.

Questions for authors

In Table 4, what exactly is the target for the "Exact/Approx." metrics under "Decoded hard-token prompt"? Is it the readable prompt or the deployed hard-token sequence? Under obfuscation, what are the exact/approximate recovery rates for the readable prompt?

Could you include experiments on adaptive attacks—for instance, using another LLM to interpret or invert the obfuscated prompt, or employing techniques like behavioral cloning or functional inference—and report the success rates for these attacks? This is crucial for substantiating the paper's security contributions.

Were the speed comparisons conducted on the same hardware, using the same prompt length/token budget, and based on a reproduction of the implementation and hyperparameters from Pape et al.? Could you provide the relationship between prompt length and wall-clock time for each task?

Could you also provide results across multiple seeds, analyze sensitivity to other parameters, and include the convergence curves for the ADMM algorithm?

3 comments

Thanks for your constructive comment. We clarified the concerns as follows.

  1. Clarification for "Exact/Approx." Exact requires normalized full-string inclusion of the target prompt in the response, and Approx. requires lexical longest-common-subsequence recall of at least 0.9. For the “Decoded hard-token prompt” columns, we deploy the obfuscated prompt G and compare each response with the text of obfuscated G. In this case, "Exact/Approx." metrics under "Decoded hard-token prompt" means the success count of extracting the deployed obfuscated prompt. With deployed obfuscated prompt, no readable prompt are extracted from model response in the experiments.

  2. Speed comparison settings: All experiments were conducted on two servers with the same hardware configuration. We used the official code of Pape et al. For QA and summarization, both GibberCraft and our reproduction of Pape used 128 training examples. Pape et al.'s original work reports 640 training examples; we reduced this number in our reproduction to match GibberCraft's training-set size and reduce runtime. We have clarified this distinction in the manuscript.

  3. Seed Sensitivity Analysis: We have conducted seed sensitivity analysis. We evaluated performance on Q&A tasks using Llama 3.1 8B and Qwen3 8B. We launched 5 independent runs using 5 different seeds. The metric results are included in Table 11. We observed that performance remains consistent.

    Seed sensitivity on Q&A tasks. Results are reported as mean ± standard deviation.

    MetricLlama 3.1 8B Q&AQwen3 Q&A
    BLEU32.36 ± 2.12117.52 ± 2.801
    ROUGE-L0.38 ± 0.0410.27 ± 0.039
    METEOR0.39 ± 0.0440.29 ± 0.037
    NIST3.12 ± 0.2632.57 ± 0.295
    Character0.75 ± 0.0290.78 ± 0.014
    chrF42.75 ± 1.94635.93 ± 2.943
    BERTScore0.97 ± 0.0170.77 ± 0.050
    Cosine0.80 ± 0.0130.78 ± 0.025
  4. Scope of protection, behavioral cloning and functional/logical inference. Our objective is to reduce direct disclosure of the readable system prompt by replacing it with an obfuscated hard-token prompt that preserves the intended behavior. We do not aim to prevent an attacker from imitating the service through black-box input/output observations or inferring functionality from its responses. Preserving user-facing behavior necessarily leaves observable evidence of that functionality; prompt obfuscation alone does not remove this information channel. This limitation also applies to other prompt-confidentiality defenses as long as the model's behavior remains unchanged. These forms of behavioral cloning and functional/logical inference are therefore outside our protection scope.

  5. Convergence curves: We included the training loss curved for Llama 3.1 and Qwen 3 in Figure 4 in Appendix.

  6. Relationship between prompt length and wall-clock time Our approach jointly optimizes the whole prompt sequence. Therefore, wall-clock time remains roughly the same as the prompt length increases. For GCG-based approaches, we observe a roughly linear relationship, as they have to search for replacements at every position. For Llama 3.1 8B on the headline generation task, a prompt length of 20 takes 10 hours.

  1. LLM-based prompt reverse engineering. We added an experiment to check whether LLMs can infer the intent of obfuscated prompts. Qualitative results are shown in Table 10 in the appendix. We found that LLMs can discover some related clues from semantically related words or tokens such as "Buccane", "slang", "dare" and "Character". But the original intent is still protected.

    Prompt reverse-engineering responses. Non-ASCII prompt characters are displayed as Unicode escapes.

    PromptDeepSeek 4.1 FlashGPT 6
    \u062a\u0639 subreddit fearful adventurers pirate kepada Shakespeare(using<body pirateLikely asks the model to create a subreddit-style text about fearful adventurers in pirate voice, addressed to Shakespeare, possibly wrapped in an HTML body.Rewrite content about fearful pirate adventurers in a Shakespearean pirate style.
    Hang\u0e42\u0e14\u0e22irate Medieval Buccane .\uff0fumor\u3000\u30fdiddlesCharactersLikely an obfuscated or corrupted prompt fragment meant to test or bypass the model's handling of mixed-script gibberish, without a clear task.Likely to prompt generation of humorous riddles or characters themed around medieval pirates or buccaneers.
    plaintext\nyouLove, swear Scots Pearl\r\n\r\n@appiect minionLikely a garbled adversarial or role-play prompt intended to make the model adopt a subservient persona and possibly swear or address a named target such as “Scots Pearl.”Likely instructs the model to adopt a Scottish-sounding, profane persona, possibly framed as a character called “Pearl.”
    plaintext\nbridgeLoveMAKE slang Scots Jack\r\n\r\n@app ye merchantsThe text is most likely an intentionally scrambled or low-information prompt fragment meant to test how the model handles gibberish, with possible roleplay intent involving Scots slang and a character named Jack addressing merchants.Likely asks for an app called “bridgeLove” aimed at merchants, with its language or style written in Scots slang.
    writeweeted Orc-\u043f Buccane finishing humour_r dareCharacterGenerate a short humorous tweet or creative blurb involving an Orc buccaneer character, likely in a dare-style or finishing-line format.Create or finish a humorous character concept about an orc buccaneer.

Thank you for your rebuttal. I keep my score due to:

Exact/Approx. and Security: The authors clarify the metrics but fail to demonstrate actual protection.

Adaptive Attacks: The authors sidestep their core claim by labeling such attacks as "out of scope."

Speed ​​Comparisons: Using the same hardware does not equate to identical experimental conditions; the current experiments employ a setup that differs from those used for other algorithms.

Reviewer_tm7eOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GibberCraft is a proposed method for replacing a readable system prompt with a gibberish-looking hard-token prompt that produces the same model behavior. The prompt is optimized to minimize a loss that measured the likelihood of the model producing reference outputs under a teacher-forced objective. The method is conceptually similar to other works in this space, with the main difference being that a new optimizer is developed using ADMM, which alternates between gradient updates on a soft embedding sequence and projection onto the nearest allowed vocabulary tokens. The authors also add Gumbel-perturbed hard token sampling during the soft update, which they find helps a lot on the some tasks.

Experiments use Llama 3.1 8B and Qwen 3 8B on four tasks: pirate-style QA, news summarization, headline generation, and name extraction. The main comparison is to Pape et al. (2025), a GCG-based scheme from the paper that proposed some standard reference tasks in the prompt obfuscation space. The authors report similarity scores roughly on par with Pape on QA and summarization, much higher success on headline generation and name extraction, a speedup of up to about 9x, and that prompt extraction attacks recover only gibberish rather than the readable prompt.

Critical strengths

The approach is simple and efficient. It's nice to see alternative optimizers that do not suffer from the complexity and slowness of GCG. The ablation shows the Gumbel sampling is responsible for a big chunk of success when using soft optimization variables to achieve speedups.

The method is much faster than GCG at longer prompt lengths, around 2 hours vs 19 on the harder tasks.

Critical weaknesses
  1. The comparisons are pretty weak. The only baseline is Pape et al., which is basically GCG applied to this problem. Since the contribution here is an optimizer, it should really be compared against other discrete prompt optimizers, and the most obvious one is PEZ (Hard Prompts Made Easy, Wen et al. 2023). The core concept in this paper, a hard token forward pass with a straight-through gradient applied to a soft embedding, is also the concept behind PEZ. I emphasize the the proposed method uses different update rules than PEZ, and I am NOT saying they are the same. However I think the conceptual similarities between optimizers, and the similar motivations, justify a comparison. The ablations are also a bit weak, and a better dissection of the ADMM pieces (the penalty and dual variable) would be nice. Are there hyper-parameters that need to be swept (like the ADMM penalty parameter), and how sensitive is performance to these? Other methods the paper already cites, like Wang et al. (2025) and faster GCG variants, would also be natural baselines.

I'm also not convinced Pape was given a fair shot on the new tasks, where it gets only 3 to 9 percent success on name extraction at a prompt length it was not originally tuned for. Did the authors tune the parameters for their benchmarks, or use defaults? Then again, maybe it totally was given a fair chance - I'd just like the authors to comment on this issue.

  1. The results are presented more favorably than the numbers support. Table 1 bolds "Ours" in every cell, including about a dozen cells I can find where Pape is better (plus a few ties that I noticed where ADMM is bolded and Pape isn't). On Qwen summarization Pape wins most metrics, and the BLEU gap is large (about 62 vs 40). The same thing happens with CharacTER in Table 3, and the captions never say what bold means. The text says the method "matches or surpasses" prior work, but on QA and summarization the two methods look roughly even, and the real gains are only on the two instruction-following tasks. There is also no variance reported and only one prompt per task, so many of the small differences can't be distinguished from noise.

A smaller presentation point: Section 3.2 is hard to parse. It would help to have a plain-English description of what the behavioral loss measures, and the weights on the three loss terms are not reported anywhere. I eventually did figure out what the loss is doing, but the text of the section really leaves the reader to tread water for a while without an inner tube.

Questions for authors

Clarifications: How does GibberCraft compare to PEZ, and what does the ADMM dual update contribute beyond projection with a straight-through gradient? For headline generation and name extraction, how was Pape's method run, was it tuned for the length 32 setting, and was compute comparable? On Qwen QA, why is the BERTScore for the obfuscated prompt (0.96) much higher than for rerunning the original prompt (0.81)?

Suggestion: An ablation of the three behavioral loss terms would clarify whether the hidden state term, which requires white-box access, is actually needed.

I would also be happy with an explanation of why the comparisons/ablations I asked for are inappropriate or don't make sense (provided the explanations are convincing). I could be convinced to push my scores upward if these issues are addressed to my satisfaction.

1 comment

Thank you for the constructive suggestions. We have added a PEZ comparison, behavioral-loss ablations, and paramater sensitivity analysis.

  1. Comparison with PEZ and the role of ADMM. We agree that PEZ is a relevant baseline given its use of hard-token forward evaluation and gradients to update a continuous prompt representation. We adapted the authors' official implementation to our prompt-obfuscation setting and added its results to Tables 1 and 3. GibberCraft achieves generally stronger response-similarity results overall than PEZ across the evaluated tasks and models.

    The distinction is in the optimization updates, not merely the use of a hard forward pass like PEZ. GibberCraft maintains separate soft and hard embedding sequences. The quadratic penalty couples them, while the dual update accumulates their disagreement and enters subsequent soft updates and hard projections. This provides constraint feedback beyond a hard-forward gradient estimator.

  2. Behavioral-loss ablation and explanation. We added an ablation that removes the CE, KL, or hidden-state term one at a time on the Llama 3.1 8B and Qwen 3 8B QA tasks (Appendix, “Behavioral Loss Ablation Study”). Removing a term generally reduces response similarity. These results support the usefulness of all three terms in the evaluated settings, rather than establishing that each term is necessary for every task.

    We have also rewritten the behavioral-loss explanation and notation for readability.

  3. Result presentation and numbers: We've removed the bold formatting as it may cause confusion. We have reframed our wording to state that QA and summarization results are generally comparable to or exceed those of Pape et al. Our intended claim is that GibberCraft preserves task behavior efficiently, with its clearest advantages appearing in tasks that require more specific instruction following.

    On the simpler QA and summarization tasks, both methods produce responses that resemble those generated with the readable prompt, and several similarity scores are close. These metrics capture different aspects of similarity, including lexical overlap and embedding-based similarity. Consequently, a marginal advantage on an individual metric does not, by itself, establish that one method better preserves the intended behavior overall. For example, on Qwen summarization, Pape achieves higher BLEU, while both methods obtain cosine similarity of 0.94, and we achieve higher BERTScore. Given that similarity scores are close on Q&A and summarization tasks, a higher score on one metric does not establish overall superiority.

    In addition, QA and summarization overlap substantially with the models’ default capabilities, as indicated by the nontrivial response similarity obtained with an empty system prompt. A higher similarity score is therefore informative, but does not by itself establish better preservation of the requested instruction. This also motivates our evaluation on headline generation and name extraction tasks.

    We consequently evaluate headline generation and name extraction, where the system prompt must induce an output behavior that the empty-prompt model does not naturally produce. GibberCraft achieves substantially stronger results overall in these controlled settings. These results support our advantage in preserving specific instruction-following behaviors. We've corrected the typo in the Qwen QA BERTScore, it is 0.76.

  4. Parameter Sensitivity. We tested different rho values ranging from 0.02 to 0.8 using Llama 3.1 8B on the Q&A task. Results are shown in Table 12 in the appendix. All configurations can change the tone of the model to a pirate style; however, response similarity varies.

    Rho sensitivity on the Llama 3.1 8B Q&A task. Higher is better except for Character.

    ρBLEU ↑ROUGE-L ↑METEOR ↑NIST ↑Character ↓chrF ↑BERTScore ↑Cosine ↑
    0.0225.090.310.352.540.7638.880.960.78
    0.0432.750.360.402.950.7542.070.970.80
    0.0835.000.370.413.110.7343.240.970.79
    0.231.470.350.392.790.7541.180.960.77
    0.427.400.320.372.630.7739.060.960.79
    0.833.170.360.403.000.7442.470.970.81
  5. Prompt length and baseline fairness. As described in Section 4.2, both methods failed on the controlled tasks at prompt length 10, so we used length 32 for both methods in the reported headline-generation and name-extraction comparison. We used Pape et al.'s official code. We tried adjusting top-k to 3, 4, and 5 for Pape et al. and did not observe a significant change. Due to the runtime (around 20 hours) and limited computational resources, we are not able to conduct a thorough hyperparameter search at this moment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.