- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript studies the personalized portfolio optimization problem. The proposed IMOPA method generates multi-objective and overall scores from an LLM, estimates preference weights through a bandit-based procedure, and trains investor-specific PPO policies. The motivation of introducing heterogeneous preferences is meaningful. However, a non-negligible gap remains between the observation model required by Proposition 1 and the fixed-score procedure used in the experiments. The validation of preference alignment is also insufficient.
Critical strengths
-
The distinction between objective-level feedback and overall preference is reasonable for describing investor heterogeneity.
-
The theoretical results are comprehensive, which seems to be correct and guarantees the effectiveness of the method.
-
The temporal train/preference/test separation and the initialization ablation help examine where the behavioral differences come from.
Critical weaknesses
-
The validity of the confidence guarantee for the implemented procedure seems to be doubtful. Remark 1 reuses a fixed score and Appendix C adds Gaussian noise. Repeated observations still contain the same . Unless this term is zero or separately controlled, the required conditional zero-mean noise assumption may not hold, and repeated use can shrink the confidence region without removing the bias. Some justifications could be helpful.
-
The experiments establish differences between LLM-defined personas, while the recovery of investor preferences seem to be unreliable. Financial returns are not an independent measure of preference alignment. The comparison is also confounded by the asset universe: Appendix D restricts each baseline to a random 15-stock subset, whereas IMOPA operates on a much larger pool. Thus, the benefit of the proposed preference-learning mechanism seems cannot be independently validated.
Questions for authors
Q1. How is justified when the same LLM score is used repeatedly? Could Proposition 1 be stated with an explicit misspecification term and the stopping rule corrected accordingly?
Q2. Proposition 1 identifies an arm in the feedback set, while Eq. (8) selects an executable policy through an additional gate. What guarantee transfers to that selected policy?
Q3. Could preference alignment be evaluated using independent feedback or known synthetic utilities, with all methods using the same asset universe and constraints?
3 comments
Response 2. Executable-policy selection and gate rule
2. Executable-policy selection and gate rule
Q2: "Proposition 1 identifies an arm in the feedback set, while Eq. (8) selects an executable policy through an additional gate. What guarantee transfers to that selected policy?"
Response: The original manuscript separates preference-optimal arm identification in Proposition 1 from executable policy selection in Eq. (8). Proposition 1 shows that Algorithm 2 produces an optimal arm based on an accurate estimate of the latent weight parameter —represented by the preference weight in Algorithm 2. Since the pair does not uniquely define a single policy in our problem formulation, we leverage them to generate two policies, and , and apply a gating rule to obtain the selected policy for each investor type . Thus, the estimation guarantee in Proposition 1 directly underpins the quality of the resulting policy. Further details are provided in Appendix B.
Following your suggestion, we also rewrite this distinction in the revised manuscript (Page 6, Lines 319–323, and Page 7, Lines 324–325) to specify the role of this rule.
Response 3. Asset-universe comparison and preference validation
3. Asset-universe comparison and preference validation
Weakness 2: "The experiments establish differences between LLM-defined personas, while the recovery of investor preferences seem to be unreliable. ... Thus, the benefit of the proposed preference-learning mechanism seems cannot be independently validated."
Q3: "Could preference alignment be evaluated using independent feedback or known synthetic utilities, with all methods using the same asset universe and constraints?"
Response: We address the comparison design and preference evaluation separately below.
“The comparison is also confounded by the asset universe: Appendix D restricts each baseline to a random 15-stock subset, whereas IMOPA operates on a much larger pool.” We explain the use of ten 15-stock subsets for the portfolio baselines. Regarding the asset datasets, IMOPA operates on the full available stock pool with a constraint of -- holdings. In contrast, DeepAries, NCMDP, and SIT do not explicitly impose such sparsity constraints. Applying these methods to the full stock pool could result in dense allocations across a large number of assets, making the out-of-sample comparison less informative. We therefore randomly sampled subsets of stocks from each dataset, evaluated the three baselines on these subsets, and reported their average performance. This design was intended to control the allocation dimension of the baselines, although it does not constitute a strictly matched asset-universe comparison with IMOPA. We have added this explanation to Appendix C (Comparison Baselines) of the revised manuscript (Page 15, Lines 772--775).
“The experiments establish differences between LLM-defined personas, while the recovery of investor preferences seem to be unreliable. Financial returns are not an independent measure of preference alignment.” As for preference alignment, in original Table 1, we evaluated investor-specific policies using five out-of-sample financial criteria: final wealth, drawdown, Sharpe ratio, turnover, and portfolio concentration. These results have showed behavioral differences consistent with the predefined investor profiles. For example, the conservative profile (PP) exhibits stronger drawdown control and lower turnover, whereas the more active profile (AA) shows greater portfolio concentration and trading activity. These results illustrate how the learned preferences translate into distinct portfolio behaviors.
To address Q3: “Could preference alignment be evaluated using independent feedback or known synthetic utilities, with all methods using the same asset universe and constraints? ” we compare IMOPA with the direct-ridge estimation, which uses the same candidate policy set, asset universe, and portfolio constraints. Their selected policies and out-of-sample financial performance were already reported in Tables F.3--F.5 of the original manuscript. Further details on the estimation procedures and the additional independent preference evaluation can be found in Common Response 1: Bandit-Based Estimation and Direct ridge. We have also evaluated preference alignment against all comparison methods reported in Table E.8 of the revised manuscript (Page 18): Direct ridge, DeepAries, NCMDP, and SIT. For each target investor profile, the independent LLM evaluator compares IMOPA's final policy with each comparison policy using the profile description and anonymized out-of-sample portfolio criteria. The evaluation protocol is described on Page 9, Lines 476--485. The Direct ridge comparison specifically provides preference evidence under the same asset universe and constraints as IMOPA.
Table E.8. Preference win rates of IMOPA against Direct ridge, DeepAries, NCMDP, and SIT (%).
| Dataset | Persona | vs. Direct ridge | vs. DeepAries | vs. SIT | vs. NCMDP |
|---|---|---|---|---|---|
| S&P 500 | PP | 14.29 | 14.29 | 25.00 | 87.50 |
| S&P 500 | FF | 66.67 | 25.00 | 50.00 | 75.00 |
| S&P 500 | II | 57.14 | 44.44 | 44.44 | 66.67 |
| S&P 500 | AA | 77.78 | 87.50 | 66.67 | 33.33 |
| MidCap100 | PP | 50.00 | 55.56 | 62.50 | 71.43 |
| MidCap100 | FF | 50.00 | 55.56 | 66.67 | 87.50 |
| MidCap100 | II | 66.67 | 55.56 | 57.14 | 42.86 |
| MidCap100 | AA | 100.00 | 77.78 | 62.50 | 55.56 |
Note. A win rate above 50% indicates that the independent LLM evaluator conditioned on the target persona, prefers the policy generated by IMOPA to the policy generated by the other algorithms.
Response 1. Fixed LLM scores and the stopping criterion of Proposition 1
We sincerely appreciate your patient examination of the theoretical assumptions and experimental comparisons. Your comments have helped us clarify the scope of the guarantees, add independent preference comparisons to illustrate out work more clearly.
1. Fixed LLM scores and the stopping criterion of Proposition 1
Weakness 1: "The validity of the confidence guarantee for the implemented procedure seems to be doubtful. ... Some justifications could be helpful."
Q1: "How is E[ξn∣Fn−1,xn]=0 justified when the same LLM score is used repeatedly? Could Proposition 1 be stated with an explicit misspecification term and the stopping rule corrected accordingly?"
Response: Proposition 1 in the original manuscript provides a stopping criterion analogous to that in linear BAI. In particular, our preference estimation is inspired by the linear best-arm identification framework of Soare et al. (2014). The confidence guarantee underlying this criterion requires the observation noise ξn(i) to be conditionally zero-mean and σ-sub-Gaussian, as assumed in Proposition 1.
Remark 1 explains the difference between our implementation and standard linear BAI in the construction of observations. Each arm receives a fixed LLM-generated scalar performance rating,
=\mathbf{x}^{\top}\mathbf{w}^{(i)} +\eta_{\mathbf{x}}^{(i)},$$ and each subsequent observation is constructed as $$y_n^{(i)} =R_{\mathbf{x}_n}^{(i)}+\epsilon_n^{(i)},$$ where $\epsilon_n^{(i)}$ is the Gaussian perturbation added at each estimation step. Since the same LLM rating is reused when an arm is selected repeatedly, its evaluation residual is also retained. Thus, the combined residual $\xi_n^{(i)}=\eta_{\mathbf{x}_n}^{(i)}+\epsilon_n^{(i)}$ is not necessarily zero-mean. This distinction is also why we refer to our procedure as a bandit-based method rather than standard linear BAI: it adapts sequential allocation and estimation to fixed offline LLM feedback. Following your comment, we have revised Remark 1 (Page 6) to clarify the two sources of observation error and explicitly state that the confidence guarantee and stopping criterion in Proposition 1 apply only when the combined residual $\xi_n^{(i)}$ satisfies the conditional zero-mean and $\sigma$-sub-Gaussian assumptions in the revised manuscript. They therefore do not provide an unconditional guarantee for the fixed-score implementation. **References** Marta Soare, Alessandro Lazaric, and Rémi Munos. Best-Arm Identification in Linear Bandits. Advances in Neural Information Processing Systems, 828–836, 2014. ---