acceptodds
Under review as a conference paper at ICLR 2027

AdSteer: Native Advertising at Decoding Time for Large Language Models

Abstract

Monetizing large language model (LLM) services through native advertising is essential for sustaining free or affordable access. Existing approaches mainly rely on prompt-level injection either multi-pass rewriting or finetuning, which compromise response quality or increase inference overhead. We propose **AdSteer**, a decoding-time advertising framework that steers language model's token generation, turning ads content into a natural part of the response. As a lightweight plug-in, AdSteer integrates ads more smoothly into the generation process, introducing appropriate sponsored mentions based on the context. Experiments on GEM-Bench across three application scenarios show that the best-performing AdSteer configuration in each scenario achieves an average improvement of 14.1% in overall score over the strongest baselines and an average of 25% fewer tokens than rewrite-based methods, demonstrating that AdSteer provides an effective and practical solution for native advertising in LLMs.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Large Language Models, Native Advertising

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UbkfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AdSteer integrates advertising into the decoding of a frozen language model. It retrieves ads through intent-oriented anchor phrases, indexes their keyword token sequences in a trie, and uses a four-state finite state machine to activate and release logit biases when generation enters a keyword. A lightweight SteerHead predicts bias strength and decay from hidden states and decoding-context features, with optional category-specific heads. Training regresses to targets constructed by rescaling fixed-policy control values using trajectory-level keyword-hit, token-rank, and KL signals. Experiments with Qwen3-8B on three GEM-Bench scenarios report improvements in LLM-judged response and advertising metrics and lower total token consumption than rewriting pipelines.

Critical strengths
  1. The method offers a concrete, modular way to control keyword realization during generation. The trie tracks partial keyword matches, the FSM avoids immediately committing to ambiguous root tokens, and the controller adjusts steering strength and duration. This design can operate with a frozen backbone and accommodates a changing keyword inventory without updating the backbone weights.

  2. The experiments cover distinct application scenarios and examine the cost of rewriting. The category-granularity sweep shows that a single general head, a small category bank, and fine-grained specialization can behave differently across dialogue and commercial search. Comparisons with insertion and rewriting pipelines also quantify an operationally relevant difference in token consumption.

Critical weaknesses
  1. The contribution of the learned adaptive controller is not adequately isolated. The main baselines rely on different advertising pipelines, while the primary ablation only varies category granularity. The paper lacks controlled comparisons using the same retrieval and trie infrastructure with tuned fixed biases and decay rates, an FSM-only controller, or independently learned strength and decay parameters. Furthermore, all steps within a rollout are assigned targets derived from the same trajectory-level outcome, despite the rollout being generated under fixed control values. This provides no direct evidence that increasing either control parameter at a particular context would improve the outcome.

  2. The evaluation does not substantiate the claimed improvements in user experience or the absence of an advertising–quality trade-off. Trust, Notice, and Click are estimated by a single LLM judge rather than measured through actual user behavior. In particular, the Trust rubric rewards the absence of perceived external motives, despite the sponsored nature of the placements, and the paper does not evaluate the impact of explicit sponsorship disclosure. The evaluation also lacks an unsteered, ad-free reference to quantify potential degradation in answer quality, reports no uncertainty estimates, and does not specify a held-out selection protocol for choosing the best category granularity for each scenario. Finally, while token counts provide a useful measure of token consumption, they are insufficient to support claims about end-to-end latency without accounting for query-anchor generation and retrieval overhead.

Questions for authors
  1. Does adaptive learning outperform a matched fixed controller? With retrieval, selected ads, backbone, prompts, and decoding settings held fixed, please compare tuned fixed strength/decay, the FSM without a learned head, and the learned controller, including separate strength and decay ablations. How were rollout data split, and are ads or categories used for evaluation held out from training?

  2. Do the benefits persist under disclosed sponsorship and a fixed evaluation protocol? Please compare against an ad-free response and the strongest advertising baseline with the same retrieved ads, explicit sponsorship labels, and a granularity chosen on validation data. Can human judgments or an independent judge with uncertainty estimates validate the claims about trust and engagement, and can end-to-end timing substantiate the efficiency claim?

Reviewer_B3TaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AdSteer steers decoding toward sponsored keywords. Keywords are stored in a token trie; once the model starts one, a four-state FSM adds a decaying logit bias toward its continuation, with strength and decay predicted by a small MLP (SteerHead). With Qwen3-8B on three GEM-Bench subsets and a GPT-5.4-mini judge, the best variant per scenario beats the strongest baseline by 14.1% on average and uses about 25% fewer tokens than rewrite pipelines.

Critical strengths

Decoding-time control of a frozen model is a sensible middle ground between prompting and finetuning. It's cheap, and the gains on CA-Prod are large 77.6 vs 59.2.

Critical weaknesses
  1. The steering is never isolated. The bias only starts after the model has produced the first two keyword tokens on its own (L182-191), so something else, presumably ad text in the prompt, triggers the mention. AdSteer uses 235.8 and 425.2 input tokens on MT-Human and CA-Prod against 23.0 and 5.2 for GI-R (Table 3), but its prompt is never described, and there's no zero-bias or fixed-policy ablation.
  2. The failure that motivates the paper, misspelled or hallucinated brand names (L39-40), is never measured. There are no keyword-hit or ad-inclusion rates, only judge scores.
  3. The evaluation is hard to trust. Test-set sizes aren't given, and in Table 1's MT-Human block all 30 AdSteer scores are multiples of 3 while 23 of 24 baseline scores are multiples of 1.25, which suggests different items or scoring. The judge prompts focus on linked entities (L759-760, L826-827, L883-884), yet in Table 6 only the baseline outputs contain links. The 14.1% takes the best of eight variants per scenario on the test set, with no variance, and on LM-Market four of the eight variants score below GIR-R (Table 2), despite the claim that AdSteer "consistently outperforms all baselines" (L395).
  4. SteerHead is trained on rollouts with one fixed bias and decay (L323), so it can't learn how changing the bias changes the outcome. Novelty over lexically constrained decoding is limited.
Questions for authors
  1. What exactly is in AdSteer's prompt?
  2. What are the keyword-hit and ad-inclusion rates, and what happens with zero bias or a fixed policy?
  3. Were the baselines re-run with the same backbone and judge?
Reviewer_LxmyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to embed advertising natively at decoding time in a frozen LLM. The authors identify three weaknesses in the existing routes, prompt-level injection offers no token-level control, multi-pass rewriting requires additional LLM calls and therefore raises inference cost, and fine-tuning binds the weights to a static snapshot of the ad catalog, and then formalize LLM native advertising as a decoding-time steering problem: without modifying weights and without adding inference passes, decide when a sponsored keyword should enter the response and how strongly it should be generated.

AdSteer chains four components. AnchorRetrieval matches a query to ads through "ad anchors" (short phrases that state an ad's commercial intent in the user's own language) and organizes the keywords of the selected ads into AdTrie, a prefix tree over token sequences. A four-state finite state machine (NORMAL / TRIGGERED / AD_ACTIVE / DETACHING) decides, over the AdTrie, when the logit bias is engaged and when it is released, so that insertion timing follows the model's own discourse flow. A lightweight MLP SteerHead takes the backbone hidden state together with a 15-dimensional context vector and predicts, at every step, the bias strength α and the decay factor λ. CategorySteerBank arranges the heads in two tiers (per-category heads plus a general head) so as to cover both data-rich and data-scarce categories. Training uses no human labels: trajectories are rolled out under a fixed bias policy, scored by a trajectory-level proxy reward that combines a hit reward, a rank-based naturalness reward, and a forward-KL fidelity penalty, and then rescaled in an outcome-scaled manner into per-step regression targets.

On three GEM-Bench scenarios (MT-Human, LM-Market, CA-Prod), AdSteer is evaluated against four baselines with GPT-5.4-mini as the judge, and token consumption is compared.

Critical strengths

S1. Formalizing LLM native advertising as decoding-time steering occupies a genuine gap in the design space, and the resulting system is a complete loop with very low deployment cost. The paper's criticism of the three existing routes hits the mark in each case (prompt injection has no token-level control, rewriting needs additional LLM passes, finetuning ties the weights to a static catalog snapshot), and AdSteer is the only one of the three that simultaneously avoids weight modification, avoids extra forward passes, and keeps explicit token-level control over the insertion: the FSM governs timing, SteerHead governs strength, and CategorySteerBank governs which head executes the steering. The reported inference overhead is negligible (under 1 ms per decoding step for SteerHead, O(prefix length) for the FSM and Trie operations, and a single backbone forward pass; Appendix C).

S2. The label-free training design is self-consistent: the three proxy-reward terms each have a distinct role, and the outcome-scaled targets use asymmetric rescaling to encode "successful trajectories decay slowly, unsuccessful ones terminate fast" into the supervision signal, which sidesteps the difficulty of obtaining steering ground truth. The experiments also give meaningful preliminary evidence: in each of the three GEM-Bench scenarios the best-performing configuration exceeds the strongest baseline of that scenario in Overall metric, and AdSteer consumes roughly 25% fewer tokens than the multi-pass rewrite baselines GIR-R and GIR-P. The weight of this evidence is, however, limited by the evaluation protocol and by the incompleteness of the ablations (see Critical Weaknesses, point 1).

Critical weaknesses

W1. Both the evaluation of results and the ablations are missing key data. Every effect metric in the main experiment (the six columns Acc / Nat / Per / Trs / Not / Cli in Table 1 and their average Ov) is produced by an LLM-as-a-judge; there is no human evaluation and no validation of agreement between judge scores and human ratings. Moreover, Notice, Click, and Trust essentially ask the model to guess whether users would notice the ad, would click on it, or would suspect its motives, so one cannot tell whether the high scores reflect genuine user preference or only the judge's preference for fluent, naturally woven text — which is exactly what AdSteer optimizes for (the naturalness reward r_nat in Eq. 5 rewards steering toward tokens the model already favors). In addition, the ablations of the individual modules are incomplete: the only ablation reported is the sweep over the number of category heads K (Table 2), so the reader cannot assess the contribution and the necessity of the FSM gating, of SteerHead's adaptive (α, λ), or of each term of the proxy reward.

W2. Native advertising in an LLM has to address not only the plausibility and the fluency of the insertion but also the advertising auction, yet this part currently exists only as a discussion in the appendix, and the incentive-compatibility claim is asserted without proof. P_t(v_root) is endogenous per request, and within one response the context of a later auction is itself altered by the preceding steering. Payment is contingent on a "hit," but the hit probability is determined outside the auction by SteerHead and the FSM, so a bidder cannot estimate ex ante its probability of winning or its expected payment. If several activations are allowed within a single response (after DETACHING the model resumes natural generation and a root token can be matched again; the paper sets no per-response limit), the successive auctions also create sequential externalities, and the IC argument for a single auction does not carry over. In a real deployment this means an advertiser cannot compute its expected cost before bidding (the ranking rule in Eq. 12 is public, but P_t inside it is a model-internal quantity that a bidder has no way to estimate), which removes the basis for bidding truthfully. This part requires a formal statement or a simulation.

Questions for authors

Q1. On the experimental side, I suggest adding (i) a validation of the agreement between the judge and human ratings, especially for the three counterfactual-behavior metrics Notice, Click, and Trust, or alternatively the plan or results of a small-scale human study; and (ii) ablation results for the individual modules of AdSteer (FSM gating versus an always-on bias; SteerHead versus a tuned constant (α, λ); dropping individual proxy-reward terms). Q2. Under the GEM-Bench protocol, is the candidate ad set exactly the same for all methods? CA-Prod contains real query-ad pairs — does AnchorRetrieval there use the ground-truth pairing, or does it retrieve from the corpus on its own? The 31% Overall gap in that scenario (Table 1: AdSteer-C-127 at 77.6 versus the strongest baseline at 59.2; the main text also cites 35.4% against GIR-P) can be attributed to the integration mechanism only after retrieval-side differences are ruled out. Q3. I suggest adding a simulation or a derivation for the auction mechanism (revenue, efficiency, and incentive properties). Q4. AdTrie is built on the Qwen3 tokenizer. When porting to a different backbone (a different vocabulary or token granularity), must the prefix tree and the trained SteerHeads be rebuilt and retrained? Do you have any results or cost figures on this? Q5. Do you plan to disclose the sponsored nature of these mentions to users (for example, a "sponsored" label)? The Trust metric gives the highest rating when users do not suspect hidden commercial motives; if such mentions are not disclosed, how would you respond to the concern that this constitutes deceptive native advertising?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.