acceptodds
Under review as a conference paper at ICLR 2027

Marrying Pricing and Advertising with LLMs

Abstract

We study a sequential pricing problem in which a seller jointly posts a price and an advertisement generated by a large language model (LLM). The seller aims to maximize revenue under an unknown product demand that depends on both decisions, while observing only whether each offer leads to a purchase. We propose an online actor-critic algorithm that combines low-rank adaptation (LoRA) of a pretrained LLM with a demand model fitted to available data. At each round, the actor generates an advertisement, and the critic estimates purchase probabilities to guide price selection. Then, the resulting feedback is used to update both the actor and the critic, with the critic's revenue estimates providing a baseline for policy gradient updates of the actor. To evaluate our approach, we develop an evaluation framework with three synthetic demand models and a demand simulator built from real-world marketplace data. Finally, we compare our algorithm with benchmarks that do not jointly optimize price selection and advertisement generation, achieving expected revenue gains over the reference policy of **5.69%**, **5.18%** and **55.96%** under the three synthetic demand models and **5.81%** under the marketplace simulator.

Primary area:
reinforcement learning
Keywords:
Pricing, Advertising, LoRA, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yJ3EOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies sequential pricing in which a seller jointly chooses a price and an LLM-generated product advertisement, using binary purchase feedback. It proposes a LoRA-based actor–critic method: a demand critic guides price selection and provides a leave-one-out baseline for KL-regularized policy-gradient updates to the advertisement generator. The method is evaluated on three synthetic demand models and a marketplace demand simulator.

Critical strengths
  • The paper formalizes the interaction between language generation and pricing, rather than treating ad generation as an isolated reward-modeling problem. Sections 2 and 3 clearly specify the information structure, LoRA-restricted actor, demand critic, and pricing rule, making the setting and optimization target concrete.

  • The actor update is well justified for the stated batch-frozen pricing policy. Proposition A.1 proves that the gradient of the surrogate in Algorithm 1, Line 11 is an unbiased estimate of the gradient of the batch-specific KL-regularized objective JτJ_\tau, including the critic-based leave-one-out baseline. The experiments also transparently evaluate expected revenue using the known synthetic demand model or frozen simulator, rather than relying only on noisy realized revenue, as described in Equation 12 and Tables 1–2.

Critical weaknesses

The experiments do not isolate whether the reported gains come from the proposed joint actor–critic procedure or from evaluating the adapted actor with a critic trained on that actor’s generated-text distribution. In Section 4, “Reference” uses the same final critic returned after online learning with the Trained actor. Because that critic has been updated on advertisements from the adapted policy, the Reference policy may be disadvantaged by critic extrapolation or less accurate pricing on reference-model texts. Conversely, the online comparison with UCB in Figures 2–3 is not matched: UCB neither uses the pretrained critic nor conditions its price on the generated advertisement, while the proposed method does. These comparisons show improvement over the paper’s stated reference protocol, but do not cleanly attribute the gain to joint optimization or establish that the actor–critic design outperforms reasonable decomposed alternatives.

This matters because the central empirical claim is that jointly adapting advertisements and prices is beneficial. A more diagnostic evaluation would include an advertisement-only LoRA baseline with a fixed or independently trained pricing rule; a critic-only, text-conditioned pricing baseline using the reference generator; and a reference-generator policy with a critic trained or updated on its own generated-text distribution under the same interaction budget. These comparisons should be reported across independent training seeds. Appendix B.1 states that the bootstrap intervals in Tables 1–2 condition on the trained policy and fixed simulator, excluding seed variation.

Questions for authors
  • In the final Trained-versus-Reference comparisons in Tables 1–2, did you evaluate a Reference generator paired with a critic trained or updated online on Reference-generated advertisements, using the same interaction budget? If so, please report the result. If not, this comparison would clarify whether the gains persist after accounting for potential critic–distribution mismatch.

  • How many independent actor–critic training seeds were run for each environment and marketplace category? Appendix B.1 states that the reported confidence intervals exclude variation across training seeds and simulator fits. Seed-level means and variation could therefore materially affect confidence in the reported effect sizes.

1 comment

We thank the Reviewer for the time and consideration in reviewing our work. The reuploaded manuscript includes the new analyses and full result tables. All four synthetic settings and three of five Avito categories have complete comparisons with three seeds and shared offline initialization. Two categories remain unfinished. Within our computational resources, we share completed evidence now to allow time for Reviewer assessment.

  1. Reference generator with its own critic. This comparison was absent from Tables 1-2. We now include it as pricing-only: the reference generator remains frozen (ητ=0\eta_\tau=0 in Algorithm 1), while its own text-conditioned critic is updated on its own generated advertisements and purchase feedback. Joint and pricing-only share the offline initialization, price grid, exploration rate, and budget of 16,000 online observations. Reference thus uses its own critic rather than Joint's final critic. Across the four synthetic settings, average normalized revenue is 0.15913 ± 0.00127 for Joint versus 0.13369 ± 0.00254 for pricing-only. Across the completed Avito categories, it is 0.21352 ± 0.00645 versus 0.20701 ± 0.00033, respectively. These are means and sample standard deviations across the three training seeds. Joint's positive average differences thus persist in the requested comparison.

  2. Independent training seeds and simulator fits. The original bootstrap intervals conditioned on one online-trained policy per setting. We now run three independent online training seeds (47, 83, 131) per method and environment/category, conditional on the shared offline initialization. The revised tables report individual-seed results, means, sample standard deviations, and paired differences. The averages above weight contexts equally within each seed. Generated ads and bootstrap resamples are not additional training replicates. Separately, we rescored frozen Avito advertisements at unchanged prices under three independently refitted simulators, using the same architecture and historical splits. Joint's positive average differences against pricing-only, sequential, and Independent persist in every refit. Its ranking against advertising-only changes. These refits assess sensitivity to simulator fitting and are not additional policy-training seeds.

  3. Decomposed alternatives and price-content interaction. We added advertising-only and sequential as restricted configurations of Algorithm 1, and an independent control combining separately trained components. Advertising-only learns ads at a fixed price with a frozen critic. Sequential trains the components in successive phases. Each receives 16,000 total online observations, divided between components for sequential and independent. Joint's average gains over Sequential and Independent are respectively 3.45% and 0.79% on synthetics, and 6.78% and 4.73% on Avito.

    When it comes to advertising-only, this one is highly competitive as its average normalized revenue is 0.15864 on synthetics and 0.21543 on Avito categories. It retains the advertising-learning component of Algorithm 1 while fixing an optimized initial price pˉ\bar p. Formally, it is recovered by setting ετ=1\varepsilon_\tau=1, ντ=δpˉ\nu_\tau=\delta_{\bar p}, and freezing critic updates (γτ=0\gamma_\tau=0), retaining pˉ\bar p at evaluation. Its average revenue exceeds pricing-only by approximately 18.7% on synthetics and 4.1% on Avito. This identifies a competitive configuration of Algorithm 1 retaining the proposed advertising-learning component. Furthermore we chose to: (1) employ a pretrained critic for every specification of Algorithm 1 and (2) sample multiple ads to allow the advertising-only agent to exploit such critic much more heavily than joint (through a Monte Carlo estimate over multiple ads, whereas joint conditions its price on each individual advertisement). In a fully online setting from scratch, however, such a price would have to be selected using a randomly initialized, unfitted critic. We opted for this approach due to time and computational constraints, as we think it is the fairest and strongest amidst others, but we commit to provide an even more insightful ablation (data scarcity progression) in the final version of the paper.

    We also added Text-crossing, extending the original synthetics with two interacting text features whose preferred combination changes with price; the demand coefficients remain unknown to the learner.

  4. UCB and information matching. UCB lacks supervised critic pretraining and advertisement-conditioned pricing, so it is secondary. Our primary comparisons use matched ablations and Independent; pricing-only shares the pretrained critic initialization and learns its own text-conditioned critic online.

Please let us know if further discussion is necessary. In case we properly addressed the Reviewer's concerns we would be grateful if the Reviewer could revise the evaluation.

Reviewer_eFSmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the joint optimization of product pricing and advertisement generation using large language models (LLMs). In a sequential pricing setting, sellers need to determine product prices and generate advertisements, while consumers' purchase probabilities depend on both the selected prices and the advertisement content. Since the underlying demand function is unknown, sellers must gradually optimize their decisions based on binary purchase feedback.

Critical strengths
  1. The paper addresses a practically relevant and well-motivated problem.

  2. The proposed method is reasonably designed and effectively integrates LLM-based advertisement generation with demand estimation.

  3. The paper introduces a critic-based leave-one-out baseline and provides corresponding theoretical analysis.

  4. The experiments cover multiple demand models and include evaluations in a simulation environment built from real-world marketplace data.

Critical weaknesses
  1. The experiments do not sufficiently demonstrate the additional benefits of jointly optimizing pricing and advertisement generation compared with optimizing them independently. The central claim of the paper is to jointly optimize product pricing and LLM-generated advertisement content. However, in the main experiments, the Trained and Reference policies use the same final critic for price selection, with the primary difference being whether the advertisement generation model is fine-tuned. Although this experimental design effectively isolates the benefits of adapting the advertisement generation policy, it does not sufficiently demonstrate that joint optimization provides additional advantages over independent optimization. In particular, the paper lacks systematic ablation studies comparing pricing-only optimization, advertisement-only optimization, and independently optimized pricing and advertisement policies.

  2. The main experiments rely on a learned demand simulator, which limits the evidence for effectiveness in real-world consumer environments. The consumer feedback in the experiments does not come from actual online interactions but is generated by a demand simulator fitted to historical marketplace data. The reported revenue improvements are also calculated based on purchase probabilities predicted by this simulator. Therefore, these results do not necessarily translate into revenue gains in real-world consumer settings.

  3. The experimental baselines are relatively limited, and the comparison with UCB involves unequal information conditions.

  4. The theoretical guarantees are relatively limited, lacking convergence or performance analysis for the overall online learning procedure.

  5. The synthetic demand models are relatively simplistic, and some of the large performance improvements may be primarily driven by predefined textual features.

Questions for authors

Please refer to Weaknesses.

1 comment

We thank the Reviewer for the time and consideration in reviewing our work. The reuploaded manuscript includes the new analyses and full result tables. All four synthetic settings and three of five Avito categories have complete comparisons with three seeds and shared offline initialization. Two categories remain unfinished. Within our computational resources, we share completed evidence now to allow time for Reviewer assessment.

  1. Additional benefits of joint optimization. We agree that the original shared critic comparison isolates advertisement adaptation and does not, by itself, establish an advantage over independent optimization. We added matched component ablations of Algorithm 1: pricing-only freezes the reference actor (ητ=0\eta_\tau=0) and learns its own critic; advertising-only fixes the price and freezes the critic; sequential updates the components in successive phases. Independent combines separately trained components of the same framework. Joint's average normalized revenue is 0.15913 on synthetics and 0.21352 on Avito categories. Relative gains over pricing-only, sequential, and independent are 19.02%, 3.45%, and 0.79% on synthetics, and 3.14%, 6.78%, and 4.73% on Avito.

    When it comes to advertising-only, this one is highly competitive as its average normalized revenue is 0.15864 on synthetics and 0.21543 on Avito categories. It retains the advertising-learning component of Algorithm 1 while fixing an optimized initial price pˉ\bar p. Formally, it is recovered by setting ετ=1\varepsilon_\tau=1, ντ=δpˉ\nu_\tau=\delta_{\bar p}, and freezing critic updates (γτ=0\gamma_\tau=0), retaining pˉ\bar p at evaluation. Its average revenue exceeds pricing-only by approximately 18.7% on synthetics and 4.1% on Avito. These comparisons identify a competitive specification of Algorithm 1 that retains the proposed advertising-learning component. Furthermore we chose to: (1) employ a pretrained critic for every specification of Algorithm 1 and (2) sample multiple ads to allow the advertising-only agent to exploit such critic much more heavily than joint (through a Monte Carlo estimate over multiple ads, whereas joint conditions its price on each individual advertisement). In a fully online setting from scratch, however, such a price would have to be selected using a randomly initialized, unfitted critic. We opted for this approach due to time and computational constraints, as we think it is the fairest and strongest amidst others, but we commit to provide an even more insightful ablation (data scarcity progression) in the final version of the paper.

  2. Access to demand data and customer experiments. As academic researchers, we lack access to controlled customer experiments and purchase data. Historical records do not directly identify d(p,x)d(p,x) for arbitrary price-ad pairs without modeling assumptions. We study sequential feedback in the specified environments. Avito now includes held-out checks against supplied probability targets and three independent refits with the same architecture and historical splits. Rescoring frozen ads at unchanged prices preserves Joint's positive average differences against pricing-only, sequential, and independent in every refit. These checks assess predictive performance and fitting sensitivity.

  3. Baseline coverage and unequal information in UCB. The primary component ablations and Independent share offline initialization and 16,000 online observations; Sequential and Independent divide this budget between components. UCB is a secondary benchmark without supervised critic pretraining or advertisement-conditioned pricing: equal online budgets do not imply equal prior information.

  4. Theoretical guarantees. Proposition A.1 proves conditional unbiasedness of the actor gradient for the batch-level KL-regularized objective with pricing fixed, even with an inaccurate or biased critic. We claim no guarantees such as convergence or regret bounds, but respectfully disagree that our analysis is insufficient: similar advertisement-generation studies also lack such guarantees: Hughes et al., KDD 2019, Wang et al., KDD 2021, Kamigaito et al., INLG 2024, Jiang et al., 2025, Chen et al., 2025, and Wang et al., KDD 2026.

  5. Synthetic demands and predefined features. Text-crossing adds two interacting text features whose preferred combination changes with price, unlike the original environments. The coefficients remain unknown to the learner. It is a controlled test of price-content interaction rather than unrestricted semantic quality.

Please let us know if further discussion is needed. If we have addressed the Reviewer's concerns, we would be grateful if the Reviewer could revise the evaluation.

Reviewer_SULROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper posts both a price and an LLM-generated ad under binary purchase feedback. A LoRA-adapted LLM (actor) writes the ad; a demand model (critic) prices it greedily and supplies a leave-one-out baseline for KL-regularized REINFORCE. Against the reference LLM priced by the same critic, it reports +5.69/+5.18/+55.96% revenue on three synthetic demands and +5.81% on an Avito-fitted simulator.

Critical strengths
  1. A clean formulation of a real gap between dynamic pricing and LLM ad generation, with a meaningful transfer test (one training product per category; 91/100 unseen products improve).
  2. Candid reporting: the appendices state their own limitations, and the numbers reproduce.
Critical weaknesses
  1. "Joint" is never tested, and the pricing half is off where it can be checked. Neither baseline optimizes the ad, and in every synthetic model the rewarded feature helps at every price, so joint and separate optimization coincide by construction. With ε = 0 the critic overprices: in price disclosure it picks about $2.99/$2.49 versus optimal prices of about $1.79/$1.84 (from Eq. 15 and Table 4); at oracle prices the 55.96% gain shrinks to about 23%, and in probit an oracle-priced Reference beats Trained.
  2. All evidence lives inside declared or fitted demand. The synthetic demand depends on one bit that the critic observes directly; the marketplace gain is scored by the same frozen MLP simulator that produced the training reward, with no ad samples, KL values, human check or simulator validation, and each setting is trained once.
Questions for authors
  1. Oracle prices and revenues next to the critic's choices — does exploration remove the overpricing?
  2. A separate-optimization baseline (ad optimized at a fixed price, then priced by the same critic), and a re-scoring of the frozen ads with an independently refitted simulator.
1 comment

We thank the Reviewer for the time and consideration in reviewing our work. The reuploaded manuscript includes the new analyses and full result tables. All four synthetic settings and three of five Avito categories have complete comparisons with three seeds and shared offline initialization. Two categories remain unfinished. Within our available computational resources, we share the completed evidence now to allow time for Reviewer assessment.

  1. Oracle pricing and exploration. We report selected and oracle prices and revenues for reference and adapted advertisements, separating advertising gains from pricing errors. Adding exploration (ε=0.1\varepsilon=0.1) reduces Probit's mean pricing regret from 0.00889 to 0.00051. Pricing regret changes very little in Logit and increases in Book. For Book, Joint's mean price remains 2.353 versus the oracle mean of 1.839. For the same advertisements, normalized revenues are 0.02682 and 0.03087, respectively. Our conclusion is that online data gathering and aggregation can help compensate for limited offline data, but do not guarantee oracle-optimal pricing, even with a pretrained critic. The residual gap in Book is non-negligible but largely driven by one seed, highlighting sensitivity across training runs rather than a systematic failure of the approach.

  2. Joint versus separate optimization. We added "pricing-only", "advertising-only", and "sequential" as specific (restricted) configurations of Algorithm 1, alongside an "independent" one separately combining trained components, all with the same online budget of 16,000 observations. The new text-crossing environment uses two interacting text features whose preferred combination changes with price, removing the price-invariant ordering of text features in the original environments. The demand coefficients remain unknown to the learner. Joint's average normalized revenue is 0.15913 on synthetics and 0.21352 on the Avito categories. Relative gains over pricing-only, sequential, and independent are respectively 19.02%, 3.45%, and 0.79% on synthetics, and 3.14%, 6.78%, and 4.73% on Avito. Advertising-only is highly competitive: its average normalized revenue is 0.15864 on synthetics and 0.21543 on the Avito categories. These are descriptive averages across three training seeds.

    Advertising-only is a strong ablation: it retains the advertising-learning component of Algorithm 1 while fixing an optimized initial price pˉ\bar p. It is recovered from Algorithm 1 by setting ετ=1\varepsilon_\tau=1, ντ=δpˉ\nu_\tau=\delta_{\bar p}, and freezing critic updates (γτ=0\gamma_\tau=0), retaining pˉ\bar p at evaluation. Its average revenue exceeds pricing-only by approximately 18.7% on synthetics and 4.1% on Avito. This supports the usefulness of advertising learning and identifies an effective instantiation of Algorithm 1. Furthermore we chose to: (1) employ a pretrained critic for every specification of Algorithm 1 and (2) sample multiple ads to allow the advertising-only agent to exploit such critic much more heavily than joint (through a Monte Carlo estimate over multiple ads, whereas joint conditions its price on each individual advertisement). In a fully online setting from scratch, however, such a price would have to be selected using a randomly initialized, unfitted critic. We opted for this approach due to time and computational constraints, as we think it is the fairest and strongest amidst others, but we commit to provide an even more insightful ablation (data scarcity progression) in the final version of the paper.

    We also optimized advertisements at a fixed price and then repriced those same frozen advertisements with the matched Joint critic, as requested. Joint's corresponding average differences are +0.78% on synthetics and −0.13% on Avito. This shared-critic diagnostic shows only small average differences between the learned advertisements.

  3. Simulator sensitivity and reporting. We rescored the frozen Avito advertisements at their original prices using three independently refitted simulators, retaining the same architecture and historical data splits. Joint's positive average differences against pricing-only, sequential, and independent persist in all three refits. Its ranking against Advertising-only changes. The revised manuscript also reports variability across training seeds, advertisement examples, sampled KL penalties, and held-out predictive validation against the supplied probability targets. A controlled consumer experiment requires access to vary advertisements and prices and observe purchases. We do not have this access, and thus focus on online learning through sequential feedback and policy adaptation in the specified demand environments.

Please let us know if further discussion is necessary. In case we properly addressed the Reviewer's concerns we would be grateful if the Reviewer could revise the evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.