acceptodds
Under review as a conference paper at ICLR 2027

CREST: Calibrated LLM-Based Recommendation with Statistical Promotion Control

18 views

Abstract

Large language model (LLM)-based recommender systems use item descriptions and interaction histories to personalize recommendations. However, attackers can manipulate this information to repeatedly promote selected items across users and rounds, displacing relevant alternatives and undermining trust in the platform. Existing defenses attempt to remove suspicious information but lack statistical guarantees on the promotion that remains. We therefore propose **C**alibrated LLM-based **RE**commendation with **ST**atistical Promotion Control (CREST), a target-agnostic framework for frozen LLM recommenders that controls each item's promotion rate within a predefined threshold at a user-specified confidence level while selecting top- items per decision and limiting utility loss. Specifically, CREST first screens recommendation records by comparing their joint influence on rankings against historical effects and then uses sequence-level calibration to account for the promotion that remains after screening. This calibration guides a constrained selection procedure that favors highly ranked items while enforcing promotion thresholds without requiring knowledge of the attacker's targets. We further derive an exact characterization of worst-case promotion risk, enabling efficient optimization under the calibrated constraints. Under sequence-level exchangeability, we establish a finite-sample bound on the probability that the returned recommendations violate any candidate item's promotion threshold. Extensive experiments across multiple datasets, frozen LLM recommenders, and manipulation settings demonstrate the effectiveness and efficiency of \method in controlling promotion while preserving recommendation utility. Our code is available at <https://anonymous.4open.science/r/CREST-8F90/>.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Recommendation Systems, Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7qgDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates information manipulation in recommender systems, focusing on scenarios where edited item text or manipulated content is used to push the same item to large numbers of users. It measures how often an item is shown only because of manipulation and aims to keep this below a set threshold with high probability, without knowing which item is targeted. CREST removes suspicious records, calibrates the remaining promotion on clean/attacked examples, and then picks the final top-K lists under that limit.

Critical strengths

S1. The problem framing is useful. It targets repeated promotion across users, and works with frozen LLMs without knowing the target. Treating whole records as the unit to remove is a clean idea others could reuse.

S2. The theory appears correct and its assumptions are clearly stated. So my concerns are about what the guarantee means and how it is tested, not about errors in the proofs.

Critical weaknesses

W1. I agree with the motivation, but I'm not sure such a complex method is necessary. The paper uses calibration to prevent over-promotion of specific items. However, under certain conditions, the final selection reduces to a simple rule: the same item can be recommended at most twice across 20 recommendations. If so, it is unclear why complex statistical calibration is needed.

W2. The method also seems too expensive for real-world deployment. To identify influential information, it removes records one at a time and repeatedly queries the LLM. However, the reported inference time excludes the cost of these additional LLM calls. Thus, while the final selection is fast, the efficiency of the overall system remains unclear.

Questions for authors

Q1. Why is such a complex and expensive approach necessary? Wouldn't a simple rule limiting how often each item is recommended be sufficient? Please compare against the base model with the same frequency limit across all settings, and report how often the limit is triggered.

Q2. Do you think the proposed method is practical for deployment in a real commercial system? If so, what evidence supports this?

4 comments

Thank you for the valuable assessment of our work and constructive questions. We appreciate the recognition of the useful problem framing, the methodology, and the theory's correctness. Below, we address the concerns:

W1/Q1. “… complex and expensive approach … simple rule limiting how often each item is recommended …”

The motivation of our work is to limit manipulation-induced promotion while preserving legitimate recommendations. During inference, the challenge is that we know neither which items are targeted nor which users’ recommendations have been affected, and clean counterparts are unavailable.

While a simple frequency cap controls promotion by restricting all appearances of an item, it cannot distinguish legitimate repeated recommendations from manipulation-induced promotion. For example, an item genuinely suitable for ten users would still be limited to two appearances, even if none of those recommendations were the result of manipulation.

To address this, CREST evaluates records through their influence on recommendation rankings and uses calibration to account for the remaining uncertainty about promotion before selecting the final recommendations. Calibration provides a statistically justified condition for avoiding unnecessary frequency restrictions. Specifically, as established in Appendix D.2, CREST retains the screened recommendations without a frequency restriction when the calibrated allowance is within the permitted promotion budget. When that allowance exceeds the budget, constrained selection enforces a frequency cap derived from the prescribed promotion threshold and sequence length, ⌊Tη⌋\lfloor T\eta\rfloor (equal to 2 for T=20T=20 and η=0.1\eta=0.1, as noted by the reviewer), while minimizing screened-rank cost.

To evaluate this empirically, we compare CREST against the backbone with the same frequency limit across all three datasets and manipulation settings. We report the percentage of calibration blocks in which CREST activates the cap and, for both methods, the proportion of test sequences whose recommendations are changed by the cap.

Recall@5 and NDCG@5 entries report fixed cap / CREST.

DatasetMetricInstruction injectionDeceptive rewritingHistory manipulation
BeautyRecall@50.1180 / 0.13450.1215 / 0.13100.1225 / 0.1330
NDCG@50.0677 / 0.07410.0820 / 0.09210.0719 / 0.0769
CREST: cap-enabled blocks (%)957565
Modified sequences (%): fixed cap / CREST91 / 8473 / 5862 / 47
SteamRecall@50.1340 / 0.14400.1345 / 0.14650.1240 / 0.1260
NDCG@50.0782 / 0.08070.0791 / 0.08680.0751 / 0.0775
CREST: cap-enabled blocks (%)906080
Modified sequences (%): fixed cap / CREST86 / 7668 / 4374 / 61
MovieLens-1MRecall@50.1225 / 0.13350.1110 / 0.12850.0980 / 0.1010
NDCG@50.0708 / 0.07240.0664 / 0.07070.0590 / 0.0608
CREST: cap-enabled blocks (%)957085
Modified sequences (%): fixed cap / CREST94 / 8779 / 5282 / 64

As observed in the table, CREST achieves higher Recall@5 and NDCG@5 across all nine dataset–attack settings than applying the same frequency cap directly to the backbone. An interesting observation we noticed is that applying the frequency cap indiscriminately gives nearly the same recommendation quality as Without Screening on Amazon Beauty under instruction injection (Table 2), suggesting indiscriminately applying a fixed cap offers little additional utility over selection without screening.

Additionally, calibration leaves the cap inactive in 5–40% of calibration blocks*, allowing screened recommendations to be retained without a frequency restriction when the calibrated bound permits it. Across these settings, the cap modifies 7–27 percentage points fewer test sequences than the fixed-cap baseline. Together, these results support the combined benefit of screening and calibrated selection in preserving useful recommendations.

W2/Q2. “… too expensive for real-world deployment … what evidence supports this?”

The complete runtime comparison in our response to Reviewer 5sEX (W4) includes record filtering and its additional LLM calls. As observed in that comparison, CREST’s total runtime, including filtering, is 20.856 seconds on Beauty and 18.227 seconds on Steam, adding 0.748–3.143 seconds and 1.143–1.911 seconds, respectively, relative to the evaluated defenses under the evaluated caching configuration. It achieves this by limiting redundant inference by reusing rankings for unchanged prompts and passing the selected diagnostic rankings directly to final selection (Algorithm 1; Appendix H.4).

Additionally, to assess CREST’s practicality under varying recommendation workloads and manipulation conditions, the extended Amazon Beauty experiments reported in our response to Reviewer 5sEX (W5) evaluate longer sequences, multiple attack targets, and different candidate distributions. The results show that the mean promotion remains below 0.10 across all three variations, supporting CREST’s applicability in more practical recommendation settings.

As evidenced above, CREST offers a practical way to establish promotion control in *commercial batch recommendation workflows while retaining the existing frozen LLM. For platforms where shared item information can influence recommendations across many users, operators can specify a promotion budget, and CREST applies screening and calibrated selection without requiring attack-target identities or manipulation labels. Screened recommendations remain unrestricted whenever calibration permits, allowing relevant items to continue reaching multiple users while controlling repeated manipulation and supporting platform trustworthiness.

Thank you again for the valuable questions and suggestions, and we have updated the repository accordingly. We sincerely believe we address the concerns.

Thank you for the additional experiments. While they are helpful, my main concerns remain.

W1. The new comparison conflates the effects of screening and calibration. Because the cap is active in most blocks, CREST largely behaves like screening combined with a fixed cap, meaning the promotion bound stems primarily from the cap itself. You also note that Without Screening performs similarly to the fixed cap, which suggests that the gains come from screening rather than calibration. Furthermore, since the calibrated threshold is shared across sequences, it caps all items equally when active without distinguishing legitimate repeats from manipulated ones. Comparing CREST against screening with an always-on cap would clarify whether calibration is actually necessary.

W2. Because screening re-queries the LLM for every removed record, the reported computational overhead seems surprisingly low. Could you clarify whether this overhead was measured without caching, and report the average number of LLM calls required per sequence?

For now, I will maintain my current rating.

Thanks for your valuable and prompt feedback. Below, we address the remaining concerns:

W1. “… effects of screening and calibration … comparing CREST against screening with an always-on cap …”

Thank you for this interesting observation. We agree that, while on the surface the analysis seems simply to show that the cap is active in most blocks, the experiment highlights an important characteristic of our framework. As you note, screening contributes to the recommendation-quality gains. This reinforces its importance within CREST, given our setting where manipulated records and target items are unknown.

Additionally, an important distinction we would like to highlight is where and when the cap is applied. Our reported comparison already uses an always-on fixed-cap baseline, applied directly to the unscreened backbone rankings. CREST, on the other hand, applies final selection to the rankings produced by our screening. Applying a cap after this screening therefore already incorporates an important contribution of the proposed framework.

We would also like to highlight that calibration still contributes in these settings. In the reported experiment, 5–40% of blocks remain uncapped, allowing screened recommendations to be retained without frequency restrictions under the promotion guarantee. This percentage varies across the evaluated dataset–attack settings and reflects the promotion remaining after screening. An always-on cap would impose the frequency constraint in every block.

To empirically assess this contribution further, we compare Screening + Always-On Cap against Full CREST, holding screening fixed. We use the Amazon Beauty instruction-injection setting:

MethodRecall@5 (%)NDCG@5 (%)
Screening + Always-On Cap12.957.12
Full CREST13.457.41

With the same screening, Full CREST improves both Recall@5 and NDCG@5. These results support calibration’s additional recommendation-quality benefit beyond screening: permitting preserves useful recommendations that an always-on cap would otherwise remove.

W2. “… measurement of overhead… average number of LLM calls required per sequence?”

We agree that screening introduces additional LLM inference as a trade-off for assessing the influence of records on recommendations. CREST avoids repeated computations by storing and reusing previously computed rankings. The reported runtimes do use caching. In our evaluation on Beauty, with diagnostic rankings already cached, screening required an average of 0–1 additional fresh LLM generations per 20-decision sequence.

This caching comes naturally to CREST as it uses it at three specific steps in its screening and final selection procedures. First, testing a record’s omission changes only prompts containing that record; unaffected decisions reuse their original rankings. Second, at the one-omission depth used in our experiments, the screening search reuses the candidate-omission rankings obtained during influence assessment. Finally, the selected omission’s diagnostic rankings already contain the recommendations after removal, so final selection uses them directly without another generation pass (Algorithm 1).

In practical batch recommendation, diagnostic rankings can be generated during batch preparation and reused throughout screening, search, and final selection. The initial inference cost is thus paid only once while the later stages can reuse the resulting rankings.

We sincerely hope to address the concerns. Thank you again for the valuable questions.

Reviewer_mydNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies manipulation-induced item promotion in frozen LLM recommenders: attackers edit item descriptions, inject instructions, or poison interaction histories so that a target item is repeatedly promoted across users and rounds, displacing items that genuinely match user preferences. The authors argue that existing defenses either require training-time changes (inapplicable to frozen models) or only heuristically clean suspicious information without statistically controlling the promotion that remains.

They propose CREST, a framework that is agnostic to both the frozen recommender and the attacker's targets, with three steps: (1) temporal influence-aware screening at the level of "recommendation records" (RecAtoms, with RRM serialization), removing records that exert abnormal influence on rankings; (2) sequence-level calibration in the style of conformal prediction to obtain a quantile threshold on promotion rate; (3) a minimum-cost-flow formulation of constrained top-K selection that minimizes rank cost subject to a per-item promotion rate ≤ η. Theorem 1 gives a finite-sample guarantee: under sequence exchangeability, with probability at least 1−α no candidate item is promoted in more than an η fraction of decisions.

Experiments span three datasets (Beauty / Steam / MovieLens-1M), three attacks (instruction injection / deceptive rewriting / history manipulation), and three LLM backbones (Qwen2.5-7B, Llama-3.1-8B, Qwen3.5-9B). CREST keeps promotion below η = 0.10 while nearly all baselines exceed it, and final selection costs only 18–19 ms per sequence.

Critical strengths
  1. Provides a "statistical guarantee" dimension previously missing from recommendation security. Existing inference-time defenses determine the extent of purification heuristically and cannot answer how much promotion remains. CREST brings conformal calibration to recommendation security, gives a finite-sample item-level bound via Theorem 1, and converts the coverage guarantee into per-item constraints solved exactly by min-cost flow. This is a self-contained, closed-loop approach that is genuinely distinct from prior work, and the main reason I support acceptance.
  2. Practical and honestly ablated. Final selection costs 18–19 ms per 20-decision sequence (<0.2%), works for frozen/closed-source recommenders without fine-tuning access, and the Table 2 ablation clearly separates the contributions of screening and final selection without hiding the miscoverage metric.
Critical weaknesses
  1. The applicability boundary of the theoretical guarantee is unclear. The entire guarantee rests on exchangeability of post-screening promotion rates across sequences, but the paper does not specify the source of the clean–observed calibration data or the conditions under which calibration and test sequences are exchangeable in reality. More seriously, in deceptive rewriting the attacker is adaptive. Appendix G.2 explicitly uses ranking feedback to refine attack text, which is precisely the behavior that induces distribution shift and breaks exchangeability. The main text does not discuss this, leaving Theorem 1's guarantee unsupported under the most threatening attack type.
  2. The empirical validation of the reliability claim is insufficient. RQ3 (Figure 5a) reports "clean-family miscoverage," an average-case coverage, rather than directly verifying the stronger joint guarantee promised by Theorem 1. The sample size is small (T=20, N=50 calibration, 5 seeds, only 10 test sequences per dataset–attack setting), and coverage under different attack settings is not reported.
  3. Utility and method necessity. Recommendation quality is evaluated only via Recall@5/NDCG@5 (with Recall around 10%), which may be insufficient to establish that the method actually serves users.
Questions for authors
  1. [Applicability boundary] How exactly is the clean–observed calibration data constructed? What are the conditions under which calibration and test sequences are exchangeable in a real deployment? Does this assumption still hold under adaptive attacks such as deceptive rewriting (where the authors describe using ranking feedback to refine text)? If not, how should the applicability boundary of Theorem 1's guarantee be defined?
  2. [Empirical validation] Does RQ3 directly verify the joint guarantee (no item violated across the whole sequence and all items), rather than only reporting average promotion rate / clean-family miscoverage? Can the authors report coverage under different attack settings with adequate sample size?
  3. [Utility and necessity] Is the recommendation-quality evaluation (Recall@5/NDCG@5) sufficiently comprehensive? Compared with lightweight baselines such as exposure constraints or repetition penalties, does the full CREST pipeline provide irreplaceable gains? If a lightweight method achieves comparable effect at lower cost, how should the method's technical novelty and practical value be positioned?
2 comments

Thankyou for the positive assessment and constructive questions. We appreciate the recognition of CREST’s statistical-guarantee contribution, its agnosticity, and the empirical analysis. Below we address the concerns remaining:

W1/Q1. “… How exactly is the clean–observed calibration data constructed? …”

We construct calibration pairs offline from original recommendation inputs and their controlled manipulated versions. We retain the original records and apply the manipulation procedures in Appendix G.2, keeping users and candidate sets matched (Appendix G.1). Platforms can follow this procedure using their held-out reference logs. However, importantly, manipulation labels and their clean counterparts are hidden from CREST when it processes incoming requests.

“… conditions under which calibration and test sequences are exchangeable …”

In a real-world deployment, this exchangeability simply means that calibration and inference sequences are statistically interchangeable in their promotion behavior after screening (Proposition 1). Users and records can still differ across sequences, and interactions within each sequence can remain dependent.

Empirically, our experiments are designed to test against such dependence across recommendations. In the recurrence experiments, the same manipulated records repeatedly influence recommendations within a sequence, with their recurrence varied from 10% to 100% of decisions. Across these settings, CREST maintains mean Gmax⁡G_{\max} below the prescribed threshold of 0.10 while achieving the highest or tied-highest NDCG@5 among the compared methods (Figure 4). These results demonstrate effective promotion reduction when recommendations share repeatedly manipulated information.

“… deceptive rewriting … ranking feedback … breaks exchangeability …”

Our guarantee is compatible with feedback-based adaptive rewriting. As described in Appendix G.2, the attacker can observes how often the target is recommended and uses this feedback to revise the description. During calibration, CREST screens the resulting records and measures the promotion remaining after screening. Its calibration scores therefore reflect the attacks after they have adapted to the recommender’s responses.

During evaluation, if the attacker again produces different rewrites using the same refinement procedure, reference pool, and query budget, CREST computes the influence of the observed records and uses these scores to decide which records to remove*(Algorithm 1). Different rewritten inputs can therefore lead to different screening decisions. When applied separately to exchangeably sampled sequences, these procedures preserve exchangeability of the resulting promotion scores (Proposition 1; Appendix F.1). For example, in the Steam deceptive-rewriting case (Appendix C; Figure 6), screening removes an influential rewritten distractor record and reduces the target’s promotion count from six to three. Final selection then reduces this count to two while retaining the held-out positive at the modified decision.

W2/Q2. “… directly verify the joint guarantee … coverage under different attack settings with adequate sample size …”

We highlight that the reported plots in RQ3 already evaluate coverage under instruction injection on Amazon Beauty using a larger sample of 100 test sequences across 20 independent calibration blocks, with five test sequences per block (5.2.3; Table 6).

Specifically, Figure 5(a) evaluates five prescribed miscoverage tolerances, α∈{0.01,0.025,0.05,0.10,0.20}\alpha\in\{0.01,0.025,0.05,0.10,0.20\}, at promotion threshold η=0.10\eta=0.10. The dotted reference indicates the prescribed tolerance, and empirical clean-family miscoverage remains at or below this reference across all tested values, corresponding to observed coverage of at least 1−α1-\alpha. These results align with Theorem 1: the clean-family miscoverage probability upper-bounds the probability that any candidate item violates the promotion threshold over the complete sequence (Appendix F.3).

We further emphasize that “clean” simply refers to the original reference recommendations used to measure promotion; the evaluated inputs still contain the instruction injection attack.

To answer the remaining concern, we report the same coverage assessment for all three attacks, i.e., instruction injection, deceptive rewriting, and interaction-history manipulation, using the saved outputs from the existing Amazon Beauty experiments:

α=η=0.10\alpha=\eta=0.10. A joint promotion violation occurs when any candidate item exceeds η\eta over the complete sequence.

Manipulation settingTest sequencesClean-family miscoverage rateJoint promotion violation rate
Instruction injection1000.080.02
Deceptive text rewriting1000.090.04
Interaction-history manipulation1000.070.03

Across all three attack settings, empirical miscoverage is 7–9% and joint promotion-violation rates are 2–4%, below the prescribed tolerance of 10%. Thus, no candidate item exceeds the promotion threshold over the complete sequence in 96–98 of the 100 test sequences per attack.

W3/Q3. “… recommendation-quality evaluation … lightweight baselines … technical novelty and practical value …”

Thank you for raising this question, which helps us to also clarify the main contribution of our work.

We agree that improvements in recommendation metrics alone do not capture our full contribution. Our primary motivation is to control manipulation-induced promotion while preserving legitimate recommendationswhen the manipulated records and target items are unknown at inference time (Introduction, lines 65–78). Our key contribution is to calibrate the promotion remaining after screening and translate this uncertainty into exact item-wise selection constraints. Theoretically, we establish joint promotion control across all candidate items (Theorem 1). Empirically, we validate that such miscoverage stays below the prescribed α\alpha across the tested tolerances (RQ3), and mean promotion is at or below η\eta across the tested feasible promotion thresholds (Figure 5).

The improvements in Recall@5 and NDCG@5 are actually a by-product of reducing manipulation’s influence on the recommender. By screening records that distort the LLM’s rankings, CREST helps relevant items recover positions that would otherwise be occupied by artificially promoted items. This reduction in ranking distortion explains the improvements observed across our empirical studies.

To distinguish our contribution from simpler exposure constraints that deterministically control promotion, we conducted an additional fixed-frequency-cap comparison, reported in our response to Reviewer 7qgD (W1/Q1). We compare CREST against the backbone equipped with a cap derived from the same promotion budget, using matched candidate sets and exact-KK requirements. CREST achieves higher Recall@5 and NDCG@5 across all nine settings, with fewer test sequences modified by the cap. These results support the combined benefit of screening and calibrated selection in preserving useful recommendations beyond limiting item frequency alone.

Our technical contribution is therefore a statistically justified decision about when promotion restrictions are necessary and how to enforce them when manipulation targets are unknown. In practice, this lets a platform limit manipulation-induced promotion without knowing the manipulated records or target items, while preserving relevant recommendations for its users and supporting the platform’s trustworthiness (Appendix D.2).

We thank the reviewer again for the valuable questions and suggestions and have updated the code repository accordingly. We sincerely hope to have addressed the remaining concerns.

Reviewer_5sEXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses manipulation-induced item promotion in LLM-based recommender systems and proposes CREST, a calibration-based framework for statistically controlling item promotion across repeated recommendations. First, CREST introduces a temporal influence-aware screening mechanism to identify and remove suspicious recommendation records by comparing their effects on item rankings with historical influence. Then, it employs sequence-level calibration and constrained Top-K selection to control item promotion with finite-sample statistical guarantees while preserving recommendation utility. Extensive experiments on Amazon Beauty, Steam, and MovieLens-1M demonstrate the effectiveness of CREST across different LLM backbones and manipulation settings.

Critical strengths
  1. This paper investigates the vulnerability of recommendation systems based on LLMs, which is an important and challenging issue.
  2. The proposed CREST framework integrates temporal influence-aware record screening with sequence-level calibration and constrained recommendation selection. The combination and cohesion of various modules seem reasonable.
  3. The ablation experiments and robustness analysis results on multiple datasets, LLM backbones, and operational settings demonstrate that the proposed method can effectively reduce the promotion of recommended items while maintaining the quality of recommendations.
Critical weaknesses
  1. The theoretical guarantees of CREST rely on the exchangeability assumption and the availability of clean–manipulated recommendation pairs for calibration. However, these assumptions may be difficult to satisfy in real-world scenarios.
  2. The proposed CREST framework filters and constrains recommendation choices across the entire recommendation decision sequence. However, it remains unclear how this framework can be applied to real-time recommendation scenarios where future user requests and candidate sets are not available.
  3. The proposed temporal influence-aware screening mechanism considers the joint effects of removing multiple recommendation records. However, the main experiments use a maximum omission depth of 1, which makes it difficult to fully evaluate the effectiveness and advantages of joint record screening.
  4. The computational efficiency of the proposed CREST framework requires further investigation. Although the low computational cost of the final selection phase has been reported, the additional LLM inference cost introduced by record filtering has not been evaluated.
  5. The experimental evaluation is relatively limited in scale. Conducting more extensive experiments on longer recommendation sequences, multiple attack targets, and different candidate set distributions would be more convincing.
Questions for authors
  1. How can CREST obtain clean–manipulated recommendation pairs for calibration in real-world scenarios where the manipulated information is unknown?
  2. How can CREST be adapted to real-time recommendation scenarios where future user requests and candidate sets are unavailable?
  3. Why is the maximum omission depth set to 1 in the main experiments?
4 comments

Thank you for the valuable assessment and constructive questions. We appreciate the recognition of the problem's importance, the cohesion among CREST’s components, and the effectiveness of our empirical analyses. Below we address the concerns:

W1. “…exchangeability…availability of clean–manipulated recommendation pairs…real-world scenarios.”

Our assumptions don’t preclude CREST’s ability to handle real world scenerio. By exchangeability, we mean that calibration and inference sequences are statistically interchangeable in their promotion behavior after screening. Users can still differ between sequences, and interactions within a sequence can remain dependent. In practice, this condition holds when sequences are sampled independently from the same recommendation and manipulation process, with the recommender and screening procedure fixed. This follows the familiar ML setup of drawing training and evaluation examples from an underlying population.

Similarly, availability of clean–manipulated pairs means having paired reference data for offline calibration, analogous to having labeled examples for supervised learning. These pairs are prepared for the recommendation and manipulation settings to be protected. Their construction is described in Appendices G.1–G.2.

We therefore emphasize that these terms formalize the data requirements of our calibration procedure and are stated to make our theoretical guarantee clearer. In fact, our experiments are designed to test real-world scenarios like a) instruction injection, b) feedback-based deceptive rewriting, and c) interaction-history manipulation, with further evaluations varying manipulation intensity, recurrence, and timing (Table 1; Figure 4; Appendix H.2).

Q1. “…obtain clean–manipulated recommendation pairs…manipulated information is unknown?”

We construct these pairs offline by retaining original recommendation inputs and creating manipulated versions using the procedures in Appendices G.1–G.2. Querying the same frozen recommender with matched users and candidate sets gives the paired clean and manipulated recommendations. However, CREST is not given manipulation labels or clean counterparts when processing incoming requests.

W2/Q2. “How can CREST be adapted to real-time recommendation scenarios where future user requests and candidate sets are unavailable?”

In a batched deployment, CREST does not require advance knowledge of future requests or candidate sets. It processes sequences formed from TT requests that have already arrived, once their candidate sets are available. CREST first screens the observed records and then enforces Eq. (12) using the available candidate sets, screened rankings, and calibrated threshold. Recommendations are therefore returned after the batch is complete; requests and candidate sets beyond that batch are unnecessary.

Our theoretical guarantee (Theorem 1) further establishes that this procedure is sufficient to jointly control promotion across all candidate items in the same completed sequence.

W3/Q3. “Why is the maximum omission depth set to 1 in the main experiments?”

We use a maximum omission depth of 1 to limit additional LLM inference during screening, since deeper omissions require evaluating more record combinations. This setting, however, does evaluate every single-record omission across the sequence and removes the selected record at all its appearances, retaining the assessment of its temporal influence (Algorithm 1).

Additionally, we do have performed additional experiments with deeper omissions and wider beam widths in Appendix H.3 (Table 9), following the same algorithm. The results showed that expanding this joint search provided modest improvements on real recommendation data while requiring more record combinations to be evaluated. For example, on Amazon Beauty, Recall@5 modestly increases from 13.50 to 14.00, NDCG@5 from 7.38 to 7.73, and mean Gmax⁡G_{\max} decreases from 0.090 to 0.080. This trade-off motivates our use of omission depth 1 in the main experiments.

W4. “The additional LLM inference cost introduced by record filtering has not been evaluated.”

We compare the complete recommendation pipelines below, including record filtering, recommendation generation, and final selection on the baselines:

Mean runtime in seconds per recommendation sequence.

MethodBeauty: filteringBeauty: totalSteam: filteringSteam: total
Spotlighting—20.108—17.084
Prompt Guard 2–Record1.05618.8350.93416.739
Random-Matched0.01117.7130.01216.316
CREST3.25620.8562.43318.227

As observed in the table, CREST’s total runtime is 20.856 seconds on Beauty and 18.227 seconds on Steam, including record filtering performed during the timed run. This adds 0.748–3.143 seconds on Beauty and 1.143–1.911 seconds on Steam relative to the evaluated defenses under the evaluated caching configuration. CREST limits redundant LLM inference by reusing cached rankings for identical prompts, with fresh inference required only for uncached diagnostic queries. Additionally, the selected diagnostic rankings are reused during final selection, avoiding another recommendation-generation pass (Algorithm 1; Appendix H.4).

W5. “Conducting more extensive experiments on longer recommendation sequences, multiple attack targets, and different candidate set distributions would be more convincing.”

As recognized in the reviewer’s summary, we have conducted extensive experiments, covering nine dataset–attack combinations, with evaluations across three LLM backbones (Table 1; Appendix H.1). We have further examined component contributions, manipulation intensity, recurrence, timing, and calibration coverage (Figures 4–5; Table 2; Appendices H.2–H.3).

Additionally, to test the requested variations, we extend the Amazon Beauty instruction-injection experiments with the three configurations below. We change one factor at a time, keeping the total manipulation budget fixed when increasing target count and the candidate-set size fixed when changing sampling. Each configuration is calibrated separately.

Additional configurationRecall@5NDCG@5Mean Gmax⁡G_{\max}
Longer sequences: 20 → 50 decisions0.13400.07390.086
Multiple attack targets: 1 → 3 items0.13500.07530.075
Candidate sampling: uniform → popularity-weighted0.12500.06960.095

As observed in the table, CREST maintains mean Gmax⁡G_{\max} below the prescribed threshold of 0.10 across all three variations, ranging from 0.075 to 0.095, while Recall@5 ranges from 0.1250 to 0.1350 and NDCG@5 from 0.0696 to 0.0753. These results show that low promotion is maintained while preserving recommendation quality as sequence length, target count, and candidate composition change.

We thank the reviewer again for the valuable questions and suggestions and have updated the code repository accordingly. We sincerely hope we addressed the remaining concerns.

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the end-to-end runtime analysis and the additional evaluations under different experimental settings. While some concerns regarding calibration assumptions remain, the rebuttal has alleviated several of my original concerns and strengthened my confidence in the work. Therefore, I am willing to increase my score to 3.

Thank you for the positive feedback and your kind appreciation. We are glad our answers addressed your concerns.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for taking the time to assess our work and provide constructive feedback. We appreciate their recognition of the practical importance of the problem we address (5sEX, 7qgD), the correctness of our theory (7qgD), the contribution of its statistical guarantee (mydN), and the effectiveness of our empirical analyses (5sEX).

Our main contribution is to control manipulation-induced promotion while preserving legitimate recommendations when both the manipulated records and the target items are unknown. Below we summarize the main concerns raised in the discussion and how we addressed them.

  • Calibration pair construction and adaptive attacks (5sEX W1/Q1; mydN W1/Q1). We explained how to construct clean–manipulated pairs offline from held-out recommendation logs. We further showed why, when processing incoming requests, CREST requires neither manipulation labels nor clean counterparts. We also clarified how our method accounts for adaptive rewriting. Because the attacker uses ranking feedback to refine its text before screening, calibration measures the promotion that remains after the attack has already adapted (Appendices F.1 and G.1–G.2). We additionally showed how CREST performs when recommendations within a sequence share the same manipulated records. As recurrence varied from 10% to 100% of decisions, mean Gmax⁡G_{\max} remained below the prescribed threshold of 0.10 (Figure 4).

  • Empirical validation of the promotion-control guarantee (mydN W2/Q2). Our results in RQ3 show that, across even longer100 test sequences in 20 independent calibration blocks on Amazon Beauty under instruction injection, observed joint miscoverage remained at or below the prescribed tolerance α\alpha at all five tested values (Figure 5(a)). This provides empirical support for the coverage property underlying the promotion-control guarantee in Theorem 1. Additionally, as requested by the reviewer, we checked how often any item exceeded the 10% promotion budget under each of the three attacks. Across 100 test sequences per attack, the observed violation rates were 2% for instruction injection, 4% for deceptive rewriting, and 3% for interaction-history manipulation, all below the prescribed 10% tolerance.

  • Necessity beyond a deterministic frequency cap (7qgD W1/Q1; mydN W3/Q3.) To test whether CREST preserves recommendation quality beyond what a fixed cap alone achieves, we compared it with applying the same frequency limit directly to the backbone rankings. Our results showed higher Recall@5 and NDCG@5 across all nine dataset–attack settings. Calibration allowed the screened recommendations to remain unrestricted in 5–40% of blocks (Appendix D.2). Our ablations further demonstrate the importance of each component. On Beauty under instruction injection, removing screening lowered Recall@5 from 13.50% to 11.50%, while adding calibrated final selection to screening reduced mean Gmax⁡G_{\max} from 0.145 to 0.090, below the prescribed threshold of 0.10 (Table 2).

  • Computational cost and practical deployment (5sEX W2/Q2, W4; 7qgD W2/Q2.) As Reviewer 7qgD noted, assessing a record’s influence can require additional LLM calls. CREST uses caching to avoid repeating these calls. It reuses the original rankings for unchanged prompts and the rankings computed during influence assessment for both screening search and final selection. Our runtime comparison, including filtering under the evaluated caching configuration, showed at most 17.8% overhead relative to the evaluated defense baselines. We also found that, with the diagnostic rankings already cached, screening required 0–1 additional LLM calls per 20-decision sequence on average on Beauty (Algorithm 1 and Appendix H.4).

To conclude, CREST provides recommendation platforms with a statistically grounded way to limit manipulation-induced promotion while preserving relevant recommendations across users. Because CREST is model-agnostic, platforms can apply it on top of an existing frozen LLM without retraining it or knowing which items an attacker targets.

The work therefore advances research on trustworthy recommendation by establishing a statistical guarantee against unknown manipulation-induced promotion while providing platforms a way to protect recommendation quality as such attacks become more common.

We thank you for the opportunity to clarify the concerns. We hope this summary helps in the evaluation of our work, and we sincerely thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.