- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies the question, "Can stochastic tokenization improve robustness?". The paper shows that the previous approach StochasTok can improve both the nomral test accuracy and the robustness in pre-training, fine-tuning, and ICL. The paper then present two approaches StochasTok-Uni and Uniform-k to improve the uniformity and support of the sampling approach in StochasTok. It then shows the Uniform-k and StochasTok-Uni has better performance than StochasTok.
Critical strengths
- The paper proposed two approaches to improve the uniformity and the support of StochasTok and empirically show that the theoretic improvement can translate to empirical improvement in the accuracy under uniform stochasity and the worst-case adversarial accuracy.
- The paper is well-written.
- The paper conduct extensive study on different configurations and on different benchmarks.
Critical weaknesses
- The paper can be improved by showing the generalization across model sizes. Especially for Figure 2a, the test accuracy drops when the StochasTok is used, but it can also be the model size limits the learning of the robustness behavior, scale up the model size can dive deep into this.
- Table 1, the results on CSQA and HellaSWAG do not show improvement at all. Why this happened and how do these two benchmarks differ from others?
Questions for authors
- Line 246-247, the notion is not defined, especially how is the used in this conditional probability.
- In Eq 2, the f(v) is not the output label because it is hard to compute a distance. so is the output space Y the probability or the logits?
- I don't see the usefulness of the section 6.1 at all, unless the paper can show that the bound is tighter than the previous bounds derived from the -ball or L0, L2, Linf adversarail settings, i.e., the stochastic tokenization can have a different bound.
3 comments
Answer to Reviewer Yq2a (2/2)
Usefulness of Section 6.1:
Our aim with the theorem was to formalize how local sensitivity over the tokenization neighbourhood affects adversarial risk. The adversarial risk (Eq. 2) is the largest change in the model’s output that an attacker can cause by choosing a tokenization within the neighbourhood. If the model’s representations change little between neighbouring tokenizations, this change, and hence the adversarial risk, is small. Theorem 6.1 bounds this sensitivity from above by the model’s Lipschitz constants. The bound itself does not distinguish between training schemes.
To assess the sensitivity, we compared the normalised distance between hidden representations of different tokenizations of the same word (Fig. 22). If the normalised distance is smaller, the hidden representations are closer and which tokenization is used affects the hidden representation of a word less. We empirically found that fine-tuning with stochastic tokenization reduces the normalised distance compared to canonical tokenization. We therefore conclude that stochastic fine-tuning reduces the model’s sensitivity to the tokenization, the quantity that controls adversarial risk in Theorem 6.1.
The worst case tokenization is measured directly by the adversarial tokenization attacks: stochastic fine-tuning reduces the adversarial risk (Fig. 6), and methods covering more of the neighbourhood achieve higher adversarial robustness (Fig. 16). We have revised Sec. 6.1 accordingly.
We hope our answer addresses the reviewer’s concerns and are happy to answer further questions.
Response
I thank authors for their responses, edits to the paper, and additional experiment results. I will retain my score.
Answer to Reviewer Yq2a (1/2)
We thank the reviewer for the positive assessment and the detailed questions. We appreciate that the reviewer recognized our extensive study and the clarity of the paper. Below, we address the concerns in detail. All corresponding changes are highlighted in the revised manuscript.
Generalization across model sizes
We provide a model-size ablation for Llama-3 with 1B, 3B, and 8B parameters (now Table 3 in Sec. 6), fine-tuned on LanguageGame and evaluated under canonical and adversarial tokenization. We present the results again in Table R5 below.
Table R5 (Model size). Canonical / adversarial accuracy (%) on LanguageGame for Llama-3 models of different sizes (αfine=0.5).
Canonically fine-tuned models become less vulnerable with scale, but remain brittle even at 8B (60.2% adversarial accuracy). Stochastic tokenization during fine-tuning improves adversarial robustness at all three model sizes, with the largest absolute gains for the smaller, initially more brittle models.
Table 2 (Table 1 in the original submission): CSQA and HellaSwag
The limited gains are not specific to CSQA and HellaSwag. In ICL, the stochastic context tokenization only results in clear improvements for LanguageGame. On the general benchmarks, the gains are small. Under uniformly sampled queries (now Table 16, Sec. I.3), the best stochastic context recovers +11.5 pp on LanguageGame, but only +0.6 to +3.8 pp on the general benchmarks, with CSQA (+1.1) and HellaSwag (+0.8) at the lower end.
With fine-tuning, however, stochastic tokenization also improves robustness on those two datasets (Tables R3–R4 below): it reduces the drop in accuracy from canonical tokenization to uniformly sampled tokenization from 12.1 to 5.8–9.3 pp on CSQA and from 4.8 to 0.4–1.4 pp on HellaSwag. We attribute the difference in robustness to the varying exposure to non-canonical tokenizations: in our fine-tuning experiments, the weights are updated using 9,741 (CSQA) and 39,905 (HellaSwag) training samples, covering a wider space of possible non-canonical tokenizations. In our ICL setting, the model is frozen and only sees K=10 examples. We now discuss this in Sec. 4.3.
Table R3 (CSQA). MCQ accuracy (%) on CSQA using Llama-8B:
Table R4 (HellaSwag). MCQ accuracy (%) on HellaSwag using Llama-8B:
Q1: Definition of P(⋅∣x;α)
P(⋅∣x;α) is the distribution over tokenizations v∈TV(x) induced by StochasTok with split budget N=α⋅∣vc∣, where α is the normalized number of splits per canonical token (Eq. 3, Sec. D.1). αICL and αeval denote this rate for the context examples and for the query, respectively.
Example: Let x= "the revolution" with vc=(
the,revolution), and assume the valid splitsthe→ (th,e) andrevolution→ (rev,olution) or (re,volution). For α=0.5, we split the canonical token sequence N=1 times. StochasTok selects a splittable token uniformly at random, and among the valid tokenizations a pair again uniformly at random:th,e,revolution)the,rev,olution)the,re,volution)A uniform scheme would assign 1/3 to each. This illustrates the sampling bias discussed in Sec. 5, which StochasTok-uni and UniK remove. We now define P(⋅∣x;α) explicitly in Sec. 2.
Q2: Output space in Eq. 2
f=A∘E maps a token sequence through the embedding and transformer block to the final linear head Wout, so f(v)∈Rm is the logit vector (Sec. G, Eq. 13). The predicted label is its argmax. We now state this in Sec. 6.1.