- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies offline pricing policy optimization under endogenous prices and potentially invalid instrumental variables. The authors propose a set of plausible demand functions and optimize worst-case revenue improvement relative to a baseline. The authors also introduce a safe policy improvement procedure and establish theoretical regret guarantees. The approach is evaluated through synthetic simulations.
Critical strengths
I find the problem to be interesting and relevant for many e-commerce applications. Relaxing the standard IV exogeneity assumption is well-motivated, the proposed optimization formulation is intuitive. I also appreciate having the formal regret bounds.
Critical weaknesses
My main concern about the paper is that all evaluations seem to rely on synthetic data only. The main simulations use relatively simple demand functions and low-dimensional feature spaces. While these experiments demonstrate the method's behavior in controlled settings, they provide very limited evidence of how it would perform in a more realistic real-world setting. Experiments with real data, more complex demand functions, and weaker instruments would substantially strengthen the paper. Also it would probably help to consider other pricing baselines that are used by the practitioners.
It also seems to me that the choice of \rho is central to the method - and it is still unclear to me how to select this parameter in practice when the degree of instrument invalidity is unknown. In the main simulations, \rho is chosen using the true violation magnitude, which if I understood this correctly would generally be unavailable in practice. A more systematic robustness analysis for this parameter and its misspecification could be helpful.
Questions for authors
(see all my questions above)
8 comments
Experimental results.
Table 1 reports mean true revenue improvements across 30 replications, with policy values evaluated using 50,000 test covariate vectors. The mean estimated radius exceeds in every setting. Our method achieves the highest mean true revenue improvement across all settings.
Table 1. Mean true revenue improvement with standard errors in parentheses, across 30 training replications.
Panel A: Instrument relevance
| Method | (, ) | (, ) | (, ) |
|---|---|---|---|
| Proposed (estimated radius) | 1.933 (0.0009) | 1.920 (0.0010) | 1.914 (0.0011) |
| OLS | 1.242 (0.0568) | -1.878 (0.0616) | -2.496 (0.0686) |
| TLS-IV | -0.463 (0.8336) | -0.115 (0.6248) | -3.070 (0.9271) |
| K-IV | 1.862 (0.0833) | -5.539 (0.9670) | -6.181 (1.0182) |
| DF-IV | -10.495 (0.0004) | -10.495 (0.0004) | -10.495 (0.0004) |
| GBDT pricing | 0.168 (0.1705) | -7.346 (0.7273) | -9.973 (0.3702) |
| NN pricing | 0.234 (0.1221) | -1.058 (0.1337) | -1.796 (0.1382) |
| Mean estimated radius | 4.6039 | 4.8934 | 5.0266 |
Panel B: Instrument invalidity
| Method | (, ) | (, ) | (, ) |
|---|---|---|---|
| Proposed (estimated radius) | 1.935 (0.0009) | 1.946 (0.0006) | 1.933 (0.0009) |
| OLS | -1.104 (0.0679) | 0.001 (0.0745) | 1.242 (0.0568) |
| TLS-IV | 0.136 (0.6790) | -0.068 (0.7599) | -0.463 (0.8336) |
| K-IV | -2.142 (0.5886) | 1.776 (0.0243) | 1.862 (0.0833) |
| DF-IV | -10.495 (0.0004) | -10.495 (0.0004) | -10.495 (0.0004) |
| GBDT pricing | -9.475 (0.5707) | -0.209 (0.1600) | 0.168 (0.1705) |
| NN pricing | -0.778 (0.0841) | -0.172 (0.0912) | 0.234 (0.1221) |
| Mean estimated radius | 3.3955 | 3.3956 | 4.6039 |
2. Real data
We use ready-to-eat breakfast cereal sales data from the IRI Infoscan database [13]. The sample contains 990 observations for 25 cereal brands in the Boston and San Francisco metropolitan markets over 20 quarters. Each observation summarizes one brand's sales in one market during one quarter. We use and recover quantity demand by dividing sales revenue by price, . After dummy coding, the covariate vector contains 24 brand indicators, one city indicator, three seasonal indicators, and two continuous covariates for quarterly advertising expenditure and the average price of other cereal brands in the same market and quarter.
We use two instruments based on the same brand's same-quarter prices in other markets: (i) the average price across New England markets outside Boston (for Boston) and across northern California markets outside San Francisco (for San Francisco); and (ii) the price in the other study market (San Francisco for Boston, and Boston for San Francisco). These instruments are relevant because the same brand shares common production costs across markets. However, as noted by Nevo and Rosen [13], regional or national promotions and shifts in consumer preferences can correlate other-market prices with unobserved local demand (), violating instrument exogeneity.
Experimental setting.
We split the observations chronologically into 590 training observations and 400 test observations. The training data are used to fit the benchmark demand models, construct the empirical ambiguity set, and learn pricing policies. Any hyperparameter selection uses training data only. The learned policies are held fixed during testing. The test data are used to construct the evaluation ambiguity set and compute empirical worst-case revenue improvements. All methods use the same linear pricing policy class, allowing prices to differ across brands, cities, and seasons. The baseline price is the training median price for the corresponding brand and city. Recommended prices satisfy and must also remain within that brand's observed training price range in that city.
Evaluation.
In this real-data application, the true demand function and counterfactual revenues are unknown. With invalid instruments, we cannot identify from the observed data. We therefore compare policies on the test set by their worst-case revenue improvement over the baseline:
\min_{f\in\mathcal F(\rho;\mathcal S_K,\mathcal T_L)} \left\\{ \widehat V_f(\widehat\pi) - \widehat V_f(\pi_0) \right\\},
where the sieve spaces satisfy and .
Table 2. Empirical worst-case revenue improvement on the held-out IRI Infoscan breakfast cereal sales data. The reported is the sensitivity radius normalized by the training-sample standard deviation of demand.
| Method | ||||
|---|---|---|---|---|
| Proposed | 18,180 | 14,903 | 9,601 | 2,494 |
| OLS | ||||
| TLS-IV | ||||
| K-IV | ||||
| DF-IV | ||||
| GBDT pricing | ||||
| NN pricing |
Results.
As shown in Table 2, the proposed method achieves the largest empirical worst-case revenue improvement at each reported radius and remains positive throughout the displayed range. Its improvement decreases from 18,180 at to 2,494 at . These results illustrate how the proposed method retains positive worst-case improvement as the specified uncertainty increases in this real-data application.
We appreciate the reviewer's careful reading and constructive suggestions. Below, we address each of your comments point by point.
Weakness 1: My main concern about the paper is that all evaluations seem to rely on synthetic data only. The main simulations use relatively simple demand functions and low-dimensional feature spaces. While these experiments demonstrate the method's behavior in controlled settings, they provide very limited evidence of how it would perform in a more realistic real-world setting. Experiments with real data, more complex demand functions, and weaker instruments would substantially strengthen the paper. Also it would probably help to consider other pricing baselines that are used by the practitioners.
Response to Weakness 1:
Thank you for this insightful comment.
We added two complementary experiments: supplementary simulations with 20 covariates, nonlinear heterogeneous demand, endogenous prices, and misspecified working demand models; and an application using real breakfast cereal sales data. The simulations separately vary instrument relevance and IV invalidity. We also include GBDT and neural-network pricing benchmarks and estimate the sensitivity radius using Section D.3 in each simulation replication.
1. Synthetic data
Data-generating mechanism.
Each replication contains n=2,000 observations. The covariates Xj∼Unif[−1,1], j=1,…,20, are mutually independent. Independently of X, we draw mutually independent U,Z,ξ∼Unif[−3,3] and V∼N(0,1). Let σ(t)=(1+e−t)−1 and define
\begin{aligned} h(\boldsymbol x) &=0.8\sin(\pi x_1)+0.6x_2x_3+0.4(x_4^2-1/3)+0.4x_5+\frac{0.4}{\sqrt{12}}\sum_{j=9}^{20}x_j,\\\\ h_2(\boldsymbol x) &=0.8x_6+0.8x_7x_8+0.3x_9,\\\\ f_0(\boldsymbol x,p) &=80+4\tanh\\{h(\boldsymbol x)\\} -110(p-1)-2(p-1)^2+0.4\sin\\!\left\\{\frac{\pi(p-1)}{0.4}\right\\} \tanh\\{h_2(\boldsymbol x)\\}. \end{aligned}
Potential demand and historical observations are generated by
Y(p)P0.3h(X)+κZ+0.8U+0.6V,Y=Y(P),=f0(X,p)+ϵ,ϵ=4U−ρ0Z+0.5ξ,=0.8+0.4σ
where ρ0=(E![E[ϵ∣X,Z]2])1/2 and κ controls instrument relevance. Specifically, the historical pricing mechanism implies Corr!(Z,log1.2−PP−0.8,,X)=1+κ2κ.
The proposed method uses the misspecified linear working demand space SK=span1,x1,…,x20,p, which does not contain the nonlinear f0. Furthermore, we set TL=span1,x1,…,x20,z,z2,z3.
Radius estimated using Section D.3 (Please see our response to Weakness 2).
We select the radius ρ using demand range and slope restrictions that mimic practical domain knowledge (such as historical capacity limits and price sensitivity from past studies [6,7]). Specifically, we supply L=40,U=120,κ=106,κ=154 as prior bounds on E[Y(p)∣X=x,P=t] and its derivative with respect to p. These bounds are valid under the data-generating mechanism and provide partial information about demand without identifying f0. In each replication, we estimate ρˉ0,cm and set ρ=ρˉ0,cm.
Additional benchmarks and policy class.
We include GBDT [9] and NN pricing [10] (implemented via scikit-learn's
GradientBoostingRegressorandMLPRegressor), which have been deployed in industrial pricing models at Zalando [11] and Meituan-Dianping [12], respectively. All methods use the same policy class \Pi= \left\\{ \pi_{\boldsymbol\theta}(\boldsymbol x) =1+\theta_0+\sum_{j=1}^{20}\theta_jx_j: \sum_{j=0}^{20}|\theta_j|\le0.2 \right\\}. The baseline policy is π0(x)=1.