acceptodds
Under review as a conference paper at ICLR 2027

Interpretable Heterogeneous Instrumented Principal Component Analysis

Abstract

Instrumented principal component analysis (IPCA) is an important category of PCA that is anchored to a set of instruments (e.g., treatments). However, standard IPCA neither supports heterogeneity nor extracts interpretable factors. To solve these problems, we propose a novel interpretable heterogeneous IPCA (HeterIPCA). It constructs the factor loadings with rich heterogeneity by a structured graph convolutional network (SGCN), achieves a nonasymptotic bound of parameter estimation, and establishes asymptotic normality of inference. Experimental results show that HeterIPCA achieves state-of-the-art performance on various evaluation metrics. This finding may shed new light on the interpretability and heterogeneity of PCA and factor analysis.

Primary area:
learning theory
Keywords:
Instrumented principal component analysis, individual heterogeneity, structured graph convolutional network, factor interpretation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_yeWvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new instrumented PCA (IPCA) approach based on graph convolutional networks (GCNs). From technical viewpoints, the method is extended from IPCA [Kelly+19], which models dynamics of temporal signals by linear factors, through non-linearly describing those factors by means of GCNs to effectively incorporate relational characteristics of micro-level instruments as well as macro-level public ones. The authors also provide theoretical analysis on the proposed model about the learning bound for the GCN-based model and asymptotic normality of its inference. In the experiments on the economical assets data, the proposed method exhibits competitive performance to the other asset-pricing models in terms of predictive performance and economic portfolio evaluation.

Critical strengths
  • In the framework of IPCA [Kelly+19], GCNs are effectively introduced to non-linearly extend factor-related representations while exploiting two types of relational information embedded in micro-/macro-level instruments.
  • The analysis in [Farrell+25] for fully-connected networks (FCNs) is extended to the proposed SGCN model for theoretical analyzing the asymptotic behaviors of the method. It theoretically supports the effectiveness of SGCN.
  • The proposed method is evaluated on real-world stock price data to produce favorable economical performance outperforming the others. Thus, the theoretical and practical stuffs are well balanced in the paper.
Critical weaknesses
  • The theoretical aspects in the paper are heavily overlapped with [Farrell+25], thus leading to limited novelty. Specifically, Sec.4 basically follows the approach of [Farrell+25]; while the analysis regarding learning bound (Sec.4.1) includes extension to cope with the GCN models, the asymptotic normality in Sec.4.2 seems to be rather directly stem from the prior work.

  • In comparison to IPCA formulation (1), apart from GCN-based modeling, the proposed approach (7) additionally leverages macro-level public instrument YY to describe α\alpha-term in IPCA, while the other comparison approaches, such as the ones reviewed in Sec.2, seem to be solely built upon the micro-level ones ZZ. Thus, the comparison experiments in Sec.5 contains concerns regarding fairness. To genuinely evaluate the efficacy of GCN-empowered modeling, the methods should be compared in the same framework using the identical input information. It also lacks clear comparison to simple baseline that replaces GCNs with feed-forward network models (MLPs); SDNN or self-only in Table A4 correspond to that?

  • The discussion about interpretability in Sec.4.3 is less convincing. While Theorem 2 provides effective bound of parameter learning in the GCN models, it is less helpful for interpreting the uniqueness of the model (parameters) in conjunction with the factors ftf_t; namely, one could conjecture that there could be another Gβ′G'_\beta for rotated factor Rft\mathcal{R}f_t through joint learning of GCNs and factors in IPCA framework. Thus, the ambiguity in the factor model is not fully resolved in the approach. Furthermore, it is hard to find out that the normality shown in Theorem 3 contributes to interpretable factors. The interpretability of the factors seems to be less analyzed even in the experiments.

  • Overall optimization for this model is not clearly shown. As to architecture shown in Fig.1, it is unclear how the Bottleneck Factor Network operates in the proposed model (7). The factor network and its input portfolio return vector ftinf_t^{in} are not explained at all in the main texts, only depicted in Fig.1.

Questions for authors
  • From theoretical aspects, please show clear novelty over the prior work [Farrell+25].
  • Please provide clear analysis/discussion regarding the interpretability, hopefully with empirical (quantitative) results.
  • Please discuss the fairness in comparison especially regarding the input data, such as YY.
  • Please detail the overall framework such as for the factor network.
7 comments
AuthorsOpenReview ↗

Reply to Reviewer [1/3]

We sincerely thank the reviewer for the constructive assessment and for recognizing the graph-based extension of IPCA, the attempt to provide statistical guarantees, and the favorable economic results. We address theoretical novelty, baseline fairness, interpretability, and implementation in turn.

1. Theoretical contribution relative to Farrell et al. (2025)

The debiasing principle used in our inference analysis employs certain ideas and concepts of the established double/debiased machine learning (DML) framework, including Farrell et al. (2025). However, Theorem 1 (the pseudo-dimension of structured graph convolutional network, SGCN) is a new theoretical result deduced in this work, as only the pseudo-dimension of the ordinary fully connected network (FCN) has been deduced before. From then on, all the theoretical results have to be reformed according to this newly-developed Theorem 1, including Theorems 2 and 3. Therefore, we actually extend the DML framework from FCN to the much more complicated network structure SGCN. Moreover, HeterIPCA (our method) studies a structured and nonlinear instrument-to-loading representation, with separate micro-instrument, public-instrument, and conditional-alpha components, instead of a stand-alone fully connected nuisance estimator.

Our theory also addresses a different statistical function class. In particular, Theorem 1 states the architecture-dependent pseudo-dimension bound

Pdim⁡(SGCN⁡)≤CdQLlog⁡(Qn),\operatorname{Pdim}(\operatorname{SGCN})\le C_d QL\log(Qn),

where QQ is the number of trainable parameters, LL the depth, and nn the graph size. Its derivation counts graph-dependent activation operations instead of simply treating the model as an ordinary fully connected network. This is an architecture-specific complexity contribution relevant to graph-based parameter sharing. Theorem 2 then pursues a finite-sample estimation bound for the structured function class. Likewise, Theorem 3 is a DML-type inference extension under its stated regularity and nuisance-rate conditions. These clarifications sharpen the difference between our graph-structured factor framework and the prior fully connected setting.

The revised Appendix A.16 also evaluates the orthogonal correction directly. Across 500 paired replications per sample size, its nominal 95% intervals attain coverage of 9797\\%, 9898\\%, and 9595\\% (Table A12). This complements the estimation analysis with finite-sample evidence for inference on the average nonlinear loading..

2. Input fairness, public instruments, and the matched MLP

2a. Fairness of the input information and the contribution of public instruments

The revised Appendix A.13 now separates the input configurations from the effect of the architecture. Table A7 specifies each model's inputs: HeterIPCA uses 94 firm characteristics, the market portfolio, and ten public predictors; IPCA uses its 94-characteristic specification; and the portfolio-based and graph-based comparators retain their respective input constructions. We then add the same ten public predictors to IPCA, CA0--CA3, and SDNN and report their predictive and economic performance at K=6K=6 in Table A8.

The augmented baselines improve on the predictive metrics, while HeterIPCA retains the strongest result on all three reported metrics. The most informative comparisons are:

ModelStock-level Rr2R_r^2 (%)Portfolio-level Rx2R_{\mathrm{x}}^2 (%)Net long-short Sharpe
HeterIPCA0.6641.5602.57
IPCA + YtY_t0.6520.4291.28
CA1 + YtY_t0.1950.7620.42
CA0 + YtY_t0.2670.4311.63

The strongest augmented portfolio-level predictor CA1 increases from 0.7350.735\\% to 0.7620.762\\%, leaving a 0.7980.798 percentage-point advantage for HeterIPCA. The strongest augmented portfolio strategy, CA0, has net Sharpe 1.631.63, compared with 2.572.57 for HeterIPCA. Augmented IPCA achieves stock-level Rr2=0.652R_r^2=0.652\\%, compared with 0.6640.664\\%. Thus, the largest gains are in characteristic-managed portfolio prediction and the economic quality of the return ranking. The matched MLP experiment below provides the additional control in which the complete input set and model capacity are identical.

AuthorsOpenReview ↗

Reply to Reviewer [2/3]

2b. A direct and fully matched MLP comparison

The self-only configuration is exactly the matched MLP control. the revised Appendix A.14 makes this equivalence explicit: setting the normalized adjacency to A^=I\hat A=I turns each graph layer into a pointwise linear map while retaining the direct characteristic path, readout, alpha branch, factor network, inputs, parameter count, and training protocol. Transferring the trained self-only weights to linear layers produces a maximum absolute difference of 0.00.0 in alpha, beta, and return predictions across K=1,…,6K=1,\ldots,6, tested on random batches and four monthly stock cross-sections.

The two models have identical parameter counts at every factor dimension, ranging from 11346 to 11591 over K=1,…,6K=1,\ldots,6. Table A9 reports all six factor dimensions over the same 420 forecast months. Averaged over K=1,…,6K=1,\ldots,6, FF49-based HeterIPCA raises stock-level Rr2R_r^2 from 0.5130.513\\% to 0.5930.593\\%, portfolio-level Rx2R_{\mathrm{x}}^2 from 0.9130.913\\% to 1.1601.160\\%, net long-short Sharpe from 2.262.26 to 2.472.47, and net long-only Sharpe from 0.980.98 to 1.071.07. Both portfolio Sharpe ratios improve at every factor dimension. At K=6K=6, the long-short Sharpe improvement is 0.250.25, with a paired 95% stationary-bootstrap interval of [0.05,0.43][0.05,0.43] based on 10000 replications and a mean block length of six months.

The graph alternatives in Table A4 provide a complementary structural comparison. Their mean portfolio-level Rx2R_{\mathrm{x}}^2 values are 1.1601.160\\% for FF49, 0.9610.961\\% for two-digit SIC, 0.9130.913\\% for self-only, 0.5580.558\\% for a density-matched random graph, and 0.3240.324\\% for the characteristic-similarity graph. In summary, the matched MLP and alternative-graph results identify economically structured industry aggregation as a productive component of the model.

3. Interpretability: algebraic rotation is not the same as economic meaning

We respectfully clarify that Section 4.3 in the previous manuscript has already explicitly acknowledged the rotation cited by the reviewer. It writes

Gβ(Zt)ft=(Gβ(Zt)R−1)(Rft).G_\beta(Z_t)f_t =\bigl(G_\beta(Z_t)R^{-1}\bigr)(R f_t).

We therefore do not overlook rotational equivalence. Our point is different: once an economically grounded nonlinear instrument-to-loading mapping has been learned, an arbitrary post-hoc rotation does not create additional economic information. It changes the basis used to express the same systematic component and is unnecessary for explaining which observable characteristics are related to the learned loading representation. This is the sense in which Section 4.3 calls an additional arbitrary rotation redundant.

This distinction can be expressed formally. Let Bt=Gβ(Zt)∈Rn×KB_t=G_\beta(Z_t)\in\mathbb{R}^{n\times K} have full column rank, and let R∈RK×KR\in\mathbb{R}^{K\times K} be invertible. The orthogonal projector onto the loading subspace is

PBt=Bt(Bt⊤Bt)−1Bt⊤.P_{B_t}=B_t(B_t^\top B_t)^{-1}B_t^\top.

Since col⁡(BtR−1)=col⁡(Bt)\operatorname{col}(B_t R^{-1})=\operatorname{col}(B_t),

PBtR−1=PBt.P_{B_t R^{-1}}=P_{B_t}.

Thus subspace-based interpretation of the characteristic-to-loading relation is exactly invariant under the reviewer's transformation. A different coordinate system cannot nullify a characteristic association computed from this projection. The argument is about the meaningfulness of the learned representation, not a claim that nonlinear functions automatically prohibit rotated functions. Theorem 2 gives estimation accuracy conditional on Assumption 1, which already imposes population identification. Theorem 3 concerns inference under additional conditions. We have revised the wording to distinguish conditional determination of factors given a fixed and full-rank loading matrix from unique identification of a jointly learned loading--factor decomposition. This precision preserves the central interpretability.

3a. Quantitative economic interpretation and stability

To verify the above interpretability with empirical (quantitative) results, we have extended the characteristic analysis in Appendix A.9 to the newly added Appendix A.17 with confidence intervals for anchored directions, random coordinate transformations, and comparisons across economic anchor sets and graph constructions.

AuthorsOpenReview ↗

Reply to Reviewer [3/3]

Characteristic structure and persistence. Table A5 gives a median Top-10 characteristic-alignment share of 0.50020.5002 for HeterIPCA, compared with 0.39780.3978 for IPCA and 0.33030.3303 for PCA. The HeterIPCA mean is 0.49530.4953, with a 95% block-bootstrap interval of [0.4828,0.5094][0.4828,0.5094]. The leading characteristics represent liquidity, trading activity, size, volatility, and market exposure. Eight of the eleven adjacent-month transitions in 2021 retain exactly the same Top-10 set. The other three have Jaccard similarity 0.8180.818. Across 600 random rotations of the fitted loading matrix, the alignment profile changes by at most 2.8×10−62.8\times10^{-6} and Top-10 membership remains unchanged. These measurements establish a concentrated and persistent economic structure in the fitted loading subspace.

Economic directions under a fixed convention. We estimate the canonical-correlation transformation and labels over 1957--2008 and apply them unchanged over 2009--2021. Table A13 reports evaluation-period correlations of 0.8020.802 for the market anchor with 95% interval [0.732,0.858][0.732,0.858], and 0.8750.875 for illiquidity with interval [0.860,0.926][0.860,0.926]. The third direction combines value and profitability associations, with correlations of 0.3330.333 and 0.4950.495. Across seven estimation windows, the leading market and illiquidity labels recur in six windows, and their top-two ordering is preserved in all seven. The economic labels and signs of the canonical directions are also retained across 500 orthogonal and 500 general invertible coordinate transformations.

External traded factors and graph sensitivity. Table A14 repeats the analysis using Mkt-RF, SMB, HML, RMW, CMA, and momentum, which are external to the factor-network inputs. The alignments and one-to-one matches are determined before 2009 and then held fixed. Across the five graph constructions, the market direction retains its Mkt-RF label, with traded-anchor evaluation correlations of 0.8660.866--0.9110.911 and cross-convention absolute correlations of 0.9250.925--0.9820.982. The third original direction consistently matches an HML-labeled traded-factor direction, with cross-convention correlations of 0.8890.889--0.9830.983. Table A15 shows that for FF49, all six matched score directions have absolute evaluation-period correlations of 0.9140.914--0.9890.989 across the two conventions.

These experiments provide economic interpretation at two explicit levels: the rotation-invariant characteristic structure of the loading space, and economically anchored directions in that space. The leading market, liquidity, and value-related associations are supported by direct quantitative evidence, and Tables A13--A15 report the behavior of every direction under the stated conventions.

4. Bottleneck Factor Network and optimization

The Bottleneck Factor Network receives a characteristic-managed portfolio-return representation and passes it through linear layers, layer normalization, and ReLU activations to generate a KK-dimensional latent-factor representation. The Micro-Graph Network generates Gβ(Zt)G_\beta(Z_t), and the Conditional Alpha Network combines representations from micro- and public-instrument branches. The predicted return is built from the systematic component Gβ(Zt)ftG_\beta(Z_t)f_t and the conditional alpha component, with squared loss as in Eqs. (7)--(9). We add such details in Section 3.1. Appendix A.4 reports Adam optimization with initial learning rate 10−410^{-4}, weight decay 10−410^{-4}, dropout 0.50.5, gradient clipping at 1.01.0, up to 200200 epochs, and early stopping after 1515 validation epochs without improvement.

Reviewer_DdcyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HeterIPCA, a graph-based extension of instrumented principal component analysis (IPCA). The motivation is that standard IPCA maps observed characteristics to factor loadings through a linear, globally shared coefficient matrix, which may be too restrictive to capture nonlinear and context-dependent heterogeneity across assets. HeterIPCA replaces this mapping with a structured graph convolutional network and adds separate components for public instruments, alpha estimation, and factor extraction. The paper further develops estimation and inference results for the proposed architecture.

Critical strengths
  • The model addresses a meaningful limitation of standard IPCA. The graph-based, nonlinear mapping from observed characteristics to loadings offers a flexible way to incorporate cross-asset relationships and heterogeneous exposures.

  • The paper attempts to provide theoretical support for a graph-based factor model. In particular, the pseudo-dimension analysis and the resulting estimation bound for SGCNs address a gap between using a structured neural architecture and analyzing its statistical complexity.

  • The empirical evaluation is broad. The paper compares HeterIPCA with several factor-model and neural-network baselines and reports predictive and portfolio results, along with additional ablations and interpretability analyses.

Critical weaknesses
  1. The factor-identification claim does not appear to follow from Theorem 2. Assumption 1 already posits that the population minimizer θ∗\theta^* is nonparametrically identified. Theorem 2 then bounds the estimation error relative to this assumed minimizer; it does not establish that the minimizer corresponds to the true latent factors or uniquely identifies the factor-loading decomposition. In the theorem’s notation, θ=(α,β,γ)\theta=(\alpha,\beta,\gamma) and Gθ(X)\mathcal{G}_\theta(X) includes the alpha and loading functions, but not the latent factor ff itself. Thus, the result appears to concern estimation of the specified functions conditional on F\mathcal{F}, rather than identification. Plus, if the underlying objective depends on the product of loadings and factors (e.g., equation (9)), rotation-equivalent pairs can yield the same fitted returns. The paper should clarify which restrictions rule out these equivalent decompositions and provide an identification argument, or temper its claims about resolving factor rotational ambiguity.

  2. The assumptions required for Theorem 3 are strong and are not shown to follow from the preceding results. The theorem assumes op(T−1/4)o_p(T^{-1/4}) L2L^2 convergence for both the nuisance-function estimates and the Hessian estimates, in addition to uniform invertibility and other regularity conditions. Theorem 2 provides a bound for the squared L2L^2 error of the nuisance functions, but it is unclear whether this bound implies the rate required by Theorem 3 in the intended high-dimensional setting. Ignoring logarithmic factors, the stated rate appears to require p>dcp>d_c to obtain an L2L^2 error smaller than T−1/4T^{-1/4}. The paper also does not establish a corresponding convergence rate for the Hessian estimator. The authors should clarify when these conditions hold for the proposed SGCN and whether they are plausible for the empirical application.

  3. Minor : line 209, σ′=sign⁡\sigma^\prime=\operatorname{sign} for σ=ReLU⁡\sigma=\operatorname{ReLU} is incorrect.

Questions for authors

Please see Weaknesses.

7 comments
AuthorsOpenReview ↗

Reply to Reviewer [1/3]

We thank the reviewer for the careful mathematical reading and for recognizing the contribution of the graph-based loading model and its complexity analysis. The revision makes the estimation target, inference conditions, and economic interpretation more explicit.

1. Theorem 2, identification, and the role of rotations

We respectfully emphasize that our previous manuscript already discusses the rotational transformation explicitly in Section 4.3:

(Gβ(Zt)R−1)(Rft)=Gβ(Zt)ft.(G_\beta(Z_t)R^{-1})(R f_t) =G_\beta(Z_t)f_t.

The intention of that discussion is not to deny a basic algebraic identity. It is to argue that Gβ(Zt)G_\beta(Z_t) is a meaningful nonlinear mapping from observed instruments and graph structure into loading representations. An additional arbitrary rotation supplies no new observable information and is not required to interpret this learned mapping. In that substantive sense, it is economically redundant, even though it is algebraically feasible.

More concretely, for a full-column-rank loading matrix BtB_t and any invertible RR, the loading subspace and its orthogonal projection satisfy

col⁡(Bt)=col⁡(BtR−1),Bt(Bt⊤Bt)−1Bt⊤=PBtR−1.\operatorname{col}(B_t)=\operatorname{col}(B_t R^{-1}), \qquad B_t(B_t^\top B_t)^{-1}B_t^\top=P_{B_t R^{-1}}.

This is exactly why our existing characteristic-level analyses in Appendix A.9 can remain meaningful without choosing a uniquely preferred latent-factor basis. Table A5 reports substantially stronger characteristic alignment for HeterIPCA (our method) than for several competing methods, and Figure A2 reports interpretable and persistent loading-subspace associations. Thereofore, even though the algebraical rotational nonuniqueness exists, the observation does show the presense of economically interpretable factor subspaces.

We thank the reviewer for pointing out that Theorem 2 establishes estimation properties under Assumption 1, rather than independently proving the global uniqueness of the entire jointly estimated factorization. We have clarified this mathematical scope and refined the corresponding notation in the revised manuscript. However, we respectfully emphasize that the algebraic feasibility of factor rotations should not be confused with the economic interpretability of the resulting representation. As explicitly discussed in Section 4.3 (including the previous and the curent versions), the loading function Gβ(Zt)G_\beta(Z_t) is constructed as a meaningful nonlinear mapping from observable characteristics ZtZ_t. This characteristic-driven structure is central to HeterIPCA: the factor loadings are determined by the learned relationship between observable characteristics and asset returns, rather than introduced as arbitrary free parameters.

We have already acknowledged in the previous manuscript that an invertible transformation can yield the algebraically equivalent factorization

(Gβ(Zt)R−1)(Rft).(G_{\beta}(Z_t)R^{-1}) (R f_t).

Nevertheless, algebraic equivalence does not imply equivalent economic interpretation. An additional arbitrary rotation is not required by our characteristic-driven modeling construction and may obscure the interpretation associated with the originally specified nonlinear loading function. Thus our argument is not that nonlinear mappings automatically eliminate every possible rotational ambiguity, but that such rotations are unnecessary for the intended economic interpretation of HeterIPCA.

1a. Quantitative interpretation of the fitted loading space

The revised Appendix A.17 evaluates economic interpretation through rotation-invariant loading-space structure and a fixed calibration-selected convention for individual directions. For an invertible RR, the loading span and fitted systematic component satisfy

col⁡(GβR−1)=col⁡(Gβ),(GβR−1)(Rft)=Gβft.\operatorname{col}(G_\beta R^{-1})=\operatorname{col}(G_\beta), \qquad (G_\beta R^{-1})(Rf_t)=G_\beta f_t.

The characteristic-alignment analysis in Table A5 gives HeterIPCA a median Top-10 alignment share of 0.5002, compared with 0.3978 for IPCA and 0.3303 for PCA, over the twelve 2021 cross-sections. Appendix A.17 adds 600 random rotations: the alignment profile changes by at most 2.8×10−62.8\times10^{-6}, and the Top-10 characteristic set remains unchanged. The same set is also persistent over time, with identical membership in eight of eleven adjacent-month transitions and Jaccard similarity 0.818 in the remaining three.

AuthorsOpenReview ↗

Reply to Reviewer [2/3]

For economic directions, the canonical transformation and labels are selected using 1957–2008 and then held fixed throughout 2009–2021. Table A13 reports evaluation-period correlations of 0.802 with the market anchor and 0.875 with illiquidity, with 95% intervals [0.732,0.858][0.732,0.858] and [0.860,0.926][0.860,0.926], respectively. The canonical directions retain their labels and signs under all 500 random orthogonal and 500 general invertible transformations examined. The coordinate convention is selected during calibration and then held fixed for the descriptive evaluation.

The revision also checks six external traded-factor anchors across five graph constructions (Table A14). The market direction consistently matches Mkt-RF, with evaluation correlations 0.866–0.911. The third original direction consistently matches an HML-labeled traded-factor direction, whose correlations with HML are 0.327–0.585 in evaluation. Under FF49, all six original/traded-factor matched directions have absolute evaluation-period correlations 0.914–0.989 (Table A15). These measurements quantify persistent characteristic structure and reproducible economic directions under stated anchor conventions, and provide concrete empirical content for the interpretation of the fitted loading space.

2. Nuisance rates needed for Theorem 3

Thanks for requesting a more explicit connection between the convergence guarantees in Theorem 2 and the inference conditions in Theorem 3. We emphasize that our theoretical formulation follows the established principles of orthogonal inference after machine learning, particularly Farrell et al. (2025), while extending the estimation framework to the substantially different setting of structured graph convolutional networks (SGCNs) and heterogeneous factor models. We clarify the relevant convergence conditions and their theoretical scope below.

First, Theorem 3 requires nuisance convergence in probability, rather than a deterministic uniform convergence guarantee. Specifically, the required condition is

∣θ^−θ∗∣L2=op(T−14),\\|\hat{\theta}- \theta^*\\|_{L_2}=o_{\mathfrak{p}}(T^{-\frac{1}{4}}),

which means

T1/4∣θ^−θ∗∣L2→p0.T^{1/4}\\|\hat{\theta}-\theta^*\\|_{L^2} \xrightarrow{\mathfrak{p}}0.

This is the standard probabilistic nuisance-rate requirement used in double machine learning (Chernozhukov et al., 2018; Farrell et al., 2025). In particular, Farrell et al. (2025, Theorem 2) explicitly impose op(T−1/4)o_{\mathfrak{p}}(T^{-1/4}) convergence conditions for both the primary nuisance estimator and the conditional-Hessian estimator to establish asymptotically valid orthogonal inference.

Our Theorem 2 provides the following high-probability squared L2L^2 estimation bound, componentwise:

+\frac{\log\log T}{T}.$$ Consequently, a **sufficient condition** for the required $o_{\mathfrak{p}}(T^{-1/4})$ rate is $$p>d_c,\qquad \log^7T\log(nT) =o(T^{\frac{p-d_c}{2(p+d_c)}}),$$ which makes one sufficient regime connecting Theorems 2 and 3. **Importantly, $p>d_c$ is sufficient but not necessary for Theorem 3**. Theorem 2 establishes a nonasymptotic upper bound rather than an exact characterization of the convergence rate of the estimator. Accordingly, the fact that this particular bound does not guarantee $o_{\mathfrak{p}}(T^{-1/4})$ convergence for $p\leq d_c$ does not imply that the required probabilistic convergence is impossible. Sharper estimation bounds or additional structural properties may yield the required rate under less restrictive conditions. **Second, the Hessian convergence and invertibility requirements are standard auxiliary conditions for orthogonal inference**. Farrell et al. (2025, Theorem 2) similarly assume an $o_{\mathfrak{p}}(T^{-1/4})$ convergence rate for the conditional-Hessian estimator and uniform invertibility, without deriving the Hessian rate directly from their preceding neural-network estimation theorem. Moreover, their discussion following Theorem 2 explicitly recognizes that these assumptions are sufficient rather than minimal and that the Hessian convergence requirement can be weakened.
AuthorsOpenReview ↗

Reply to Reviewer [3/3]

This separation is theoretically meaningful. Neyman orthogonality eliminates first-order sensitivity to nuisance-estimation errors, so that under appropriate regularity conditions, the remaining bias is governed by higher-order terms. The T−1/4T^{-1/4} threshold provides a convenient sufficient rate for controlling these terms at the T\sqrt{T} inference scale. As further explained by Farrell et al. (2025, Section 3.3), the conditional Hessian can be treated as a separate nuisance function, and estimated with observation-level derivatives and conditional regression with appropriate sample splitting. We have emphasized below Theorem 3 in the revised manuscript that our Hessian-rate requirement is an explicit inference assumption, rather than a conclusion claimed to follow automatically from Theorem 2.

Third, the methodological contributions of HeterIPCA remain distinct from those of Farrell et al. (2025). While Farrell et al. directly employ the existing pseudo-dimension of fully connected networks (FCNs), HeterIPCA establishes the pseudo-dimension of SGCNs (Theorem 1), which is a new theoretical result. From then on, all the theoretical results have to be reformed according to this newly-developed Theorem 1, including Theorems 2 and 3. Theorem 2 derives the corresponding nonasymptotic estimation guarantee incorporating graph-architecture complexity. Theorem 3 then adapts the established orthogonal-inference construction to the proposed factor-model formulation. Therefore, we actually develop a new double machine learning (DML) framework for the much more complicated network structure SGCN. Moreover, HeterIPCA (our method) studies a structured and nonlinear instrument-to-loading representation, with separate micro-instrument, public-instrument, and conditional-alpha components, rather than a stand-alone fully connected nuisance estimator.

In summary, Theorems 2 and 3 provide complementary theoretical guarantees: Theorem 2 establishes an original nonasymptotic estimation bound for SGCNs and identifies a sufficient regime for achieving the nuisance convergence rate, while Theorem 3 establishes asymptotically valid orthogonal inference under explicitly stated probabilistic convergence and regularity conditions. This separation is consistent with established double machine learning theory, without diminishing the distinct graph-structured estimation contributions of HeterIPCA. We have made these connections and their scope more explicit in the revised manuscript.

3. ReLU derivative

We thank the reviewer for identifying the typo. For σ(u)=max⁡0,u\sigma(u)=\max\\{0,u\\}, the correct almost-everywhere derivative is

σ′(u)=Iu>0,\sigma'(u)=\mathbb{I}\\{u>0\\},

with an appropriate subgradient convention at u=0u=0. We have revised this issue throughout the paper.

Reviewer_zWC3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HeterIPCA, an extension of instrumented principal component analysis that incorporates nonlinear and heterogeneous factor loadings through a structured graph convolutional network. The framework combines firm-specific characteristics, industry-level graph relationships, and macroeconomic instruments to model latent factors and abnormal returns. The reported results show improvements in out-of-sample predictive R² and portfolio Sharpe ratios, supported by graph-structure ablations and interpretability analyses.

Critical strengths
  • Meaningful integration of structured factor modeling and graph learning. HeterIPCA combines firm-specific characteristics, industry relationships, and macroeconomic instruments within a nonlinear factor model. The architecture provides a potentially useful framework for capturing heterogeneous factor exposures in asset pricing.

  • Promising empirical results and comprehensive evaluations. The method reports improvements in out-of-sample predictive R² and portfolio Sharpe ratios across different factor dimensions. The graph-structure ablations, turnover analysis, and characteristic-level interpretation experiments provide additional evidence of its practical utility.

Critical weaknesses
  • The central factor-identifiability claim is theoretically unjustified. The paper argues that nonlinear instrument-dependent loadings eliminate the rotational ambiguity of IPCA. Let B(Z)B(Z) denote the nonlinear factor loading matrix and ff the latent factors. For any invertible matrix RR, replacing B(Z)B(Z) with B(Z)RB(Z)R and ff with R−1fR^{-1}f leaves the factor component unchanged, since B(Z)RR−1f=B(Z)fB(Z)RR^{-1}f = B(Z)f. Thus, nonlinearity alone does not eliminate factorization ambiguity. Furthermore, Assumption 1 explicitly presumes nonparametric identification, making the subsequent identification argument circular. The rotation-dependent axis interpretation acknowledged in Appendix A.9 further weakens the main claim.

  • The theoretical guarantees contain substantial gaps. Theorem 2 applies a neural-network approximation result to SGCNs without establishing that the graph-constrained architecture retains the required approximation capacity. The analysis also conflates global network parameters with input-dependent parameter functions. More seriously, the asymptotic expansion in Eq. (21) appears incorrectly normalized by the number of folds rather than the sample size. The DML argument additionally treats factor observations as given, although the proposed architecture learns latent factors jointly with the loading model. These issues undermine the claimed convergence and inference guarantees.

  • The graph construction may undermine the intended individual heterogeneity. The FF49 graph connects all firms sharing an industry label. Under the stated normalized GCN propagation, each industry forms a complete graph whose aggregation matrix averages all within-industry features. Consequently, node representations within an industry become identical after one layer unless explicit firm-level bypasses exist. The paper does not clarify how the proposed architecture preserves individual characteristics under this construction, or how this limitation affects its theoretical approximation claims.

  • The experiments do not adequately validate the statistical inference claims. The main evaluation focuses on one U.S. equity panel and emphasizes predictive R² and portfolio Sharpe ratios, neither of which directly validates asymptotic normality or confidence-interval coverage. The paper provides limited information about the precise temporal train/validation/test protocol and uncertainty in the main portfolio comparisons. Moreover, the transaction-cost accounting and the distinction between interpretable loading subspaces and individually identified factors require clarification.

Questions for authors
  • What explicit constraints eliminate rotational equivalence between nonlinear factor loadings and latent factors? Why does Theorem 2 establish identification rather than estimation consistency under an assumed identification condition?

  • How is the global network-parameter formulation reconciled with the input-dependent nuisance functions used in the DML derivation? Can the authors correct Eq. (21) and provide a valid asymptotic expansion with the appropriate sample-size normalization?

  • Given the complete within-industry FF49 graph, how does the GCN retain differences between firms belonging to the same industry? Are there residual connections or firm-specific paths absent from Eq. (10)?

  • What justifies applying the FCN approximation theorem to the SGCN architecture? How are estimation errors from the jointly learned latent factors incorporated into Theorems 2 and 3?

  • Can the authors provide empirical confidence-interval coverage or Monte Carlo validation of the inference procedure, together with precise temporal split details, uncertainty estimates for portfolio Sharpe ratios, and the transaction-cost calculation?

10 comments
AuthorsOpenReview ↗

Reply to Reviewer [1/5]

We thank the reviewer for the detailed technical assessment and for recognizing the value of combining nonlinear factor modeling, industry relationships, and economic instruments. The revised analysis and experiments address the questions as follows.

1. Rotation equivalence, interpretation, and what Section 4.3 actually claims

The reviewer writes that B(Z)f=(B(Z)R)(R−1f)B(Z)f=(B(Z)R)(R^{-1}f) for invertible RR. We have already explicitly acknowledged the same algebraic transformation in Section 4.3 of the previous manuscript, using the equivalent notation

Gβ(Zt)ft=(Gβ(Zt)R−1)(Rft).G_\beta(Z_t)f_t =(G_\beta(Z_t)R^{-1})(R f_t).

Therefore, it would be inaccurate to characterize our manuscript as having failed to consider rotation. Our intended position is more nuanced: the learned Gβ(Zt)G_\beta(Z_t) is a nonlinear mapping anchored to observable characteristics and economically structured neighborhood information. Applying an additional arbitrary R−1R^{-1} is a possible change of latent coordinates, but it adds no economic information and is unnecessary to interpret the instrument-associated representation already learned. The word ``redundant'' in Section 4.3 refers to this economic and explanatory redundancy, not algebraic impossibility.

Indeed, our interpretation can be stated without choosing any factor axis. For a full-column-rank Bt=Gβ(Zt)B_t=G_\beta(Z_t), define

Pt=Bt(Bt⊤Bt)−1Bt⊤.P_t=B_t(B_t^\top B_t)^{-1}B_t^\top.

For any invertible RR,

col⁡(BtR−1)=col⁡(Bt),PBtR−1=Pt.\operatorname{col}(B_t R^{-1})=\operatorname{col}(B_t), \qquad P_{B_t R^{-1}}=P_t.

Hence characteristic alignment with the loading subspace is exactly invariant to the reviewer's rotation. This is not just a verbal distinction: Appendix A.9 (previous manuscript) measures rotation-invariant characteristic alignment. Table A5 reports a median Top-10 share of 0.50020.5002 for HeterIPCA, against 0.39780.3978 for IPCA, 0.41600.4160 for SDNN, and 0.48260.4826 for CA3. Figure A2 identifies liquidity, size, and risk characteristics and reports fixed transformation associations for market exposure (0.800.80) and illiquidity (0.880.88). Crucially, Appendix A.9 already states that individual axis labels are transformation-dependent. Rotational freedom is therefore consistent with the empirical interpretation we demonstrate.

Algebraic equivalence does not imply equivalent economic interpretation. HeterIPCA constructs its nonlinear loading function Gβ(Zt)G_\beta(Z_t) directly from economically meaningful observable characteristics, thereby providing a characteristic-driven representation of heterogeneous factor exposures. Although alternative loading--factor decompositions can be obtained through invertible rotations, these transformations are not required by our modeling construction and may obscure the economic interpretation associated with the originally specified nonlinear loading function. Importantly, our argument is not that nonlinearity automatically eliminates all algebraic rotational ambiguities, but rather that such rotations are unnecessary for the intended economic interpretation of HeterIPCA. We therefore distinguish the mathematical equivalence of alternative factor representations from the economic interpretability of our characteristic-driven specification.

Assumption 1 imposes identification of the population structural-function target for the factor representation used in the theoretical analysis. Theorem 2 establishes estimation accuracy for that target. Economic interpretation is provided through the rotation-invariant loading subspace and through individual directions defined under the stated anchor convention.

1a. Quantitative interpretation of the loading subspace and economically anchored directions

The revised Appendix A.17 evaluates the rotation-invariant loading subspace and assigns economic coordinates through a fixed anchor convention. This makes the object of each interpretation explicit. Under an invertible change of coordinates, the loading column space and fitted systematic return component remain unchanged.

At the subspace level, the median Top-10 characteristic-alignment share is 0.5002, compared with 0.3978 for IPCA and 0.3303 for PCA (Table A5). The leading characteristics describe liquidity, trading activity, firm size, volatility, and market exposure. Across 600 random rotations, the alignment profile changes by at most 2.8×10−62.8\times10^{-6} and the Top-10 set remains unchanged. In 2021, eight of eleven adjacent-month transitions retain the identical Top-10 membership. The remaining three have Jaccard similarity 0.818.

AuthorsOpenReview ↗

Reply to Reviewer [2/5]

At the direction level, the canonical transformation, labels, and orientation are calibrated in 1957–2008 and then held fixed for the descriptive evaluation in 2009–2021. Table A13 reports evaluation correlations of 0.802 [0.732, 0.858] with the market anchor and 0.875 [0.860, 0.926] with illiquidity. These directions retain their dominant economic labels. The third direction has value and profitability associations, with correlations 0.333 and 0.495, respectively. These are economic coordinates under the stated anchor convention.

Tables A14–A15 further examine external traded-factor anchors and five graph constructions. The traded factors are external to the factor-network inputs, and transformations, labels, and one-to-one matches are fixed before the 2009–2021 evaluation. The market direction has evaluation correlations of 0.866–0.911 with Mkt-RF across the five graphs and absolute cross-convention correlations of 0.925–0.982. The third original direction matches an HML-labeled traded-factor direction in all five graphs, with cross-convention correlations of 0.889–0.983. For FF49, all six matched directions have absolute evaluation correlations of 0.914–0.989 across the two anchor conventions. These results quantify stability of the characteristic structure and leading economic associations while keeping each directional label tied to its stated economic convention.

2. Clarification of Theoretical Guarantees and Asymptotic Inference

We thank the reviewer for the careful examination of our theoretical analysis. Several distinctions should be made more explicit between network parameterization, structural-function estimation, and statistical inference. Our theoretical framework builds upon the structural deep learning and double machine learning (DML) methodology of Farrell et al. (2025), while extending the estimation analysis to the graph-structured setting. We address the specific concerns below.

2a. Global network parameters versus input-dependent structural functions.

The globally shared network parameters and the input-dependent structural functions are two distinct mathematical objects, and their distinction is fundamental to our formulation. Specifically, let θ=(α,β,γ)\theta=(\alpha,\beta,\gamma) be the globally shared learnable network weights. These weights parameterize the input-dependent structural function

Gθ(X):=(Γα(Gβ(Z),Gγ(Y)),Gβ(Z)).\mathscr{G}_{\theta}(X):=(\Gamma_{\alpha}(G_{\beta}(Z),G_{\gamma}(Y)),G_{\beta}(Z)).

Thus, although θ\theta is globally shared, Gθ(X)\mathscr{G}_{\theta}(X) varies with the observed instruments and represents the heterogeneous alpha and factor-loading functions.

Importantly, the statistical objects analyzed in Theorems 2 and 3 are the structural functions generated by the networks, rather than the internal network weights themselves. Correspondingly, the derivatives in the DML orthogonal score are taken with respect to (w.r.t.) the finite-dimensional structural parameter values, evaluated at the network outputs, rather than w.r.t. the complete network-weight vector. This distinction follows the structural parameter-function approach of Farrell et al. (2025, Sections 3.2–3.3 and Remark 5.2). We have made the distinctions between the global network weights and their input-dependent functional outputs more prominent in this revision.

2b. Approximation capacity of the SGCN architecture.

We respectfully clarify that the SGCN approximation error is explicitly incorporated into our theoretical analysis in the previous manuscript, rather than being overlooked. Specifically, Eq. (46) defines the best approximation in the SGCN function class and the corresponding approximation error εT\varepsilon_T. This architecture-dependent approximation error is carried through the subsequent estimation analysis and explicitly appears in many subsequent equations Eqs. (51)--(83). This follows the general network estimation framework of Farrell et al. (2021, Theorem 2), which accommodates different feedforward architectures through their corresponding approximation errors. Moreover, Theorem 1 establishes an SGCN-specific pseudo-dimension bound based on the complexity analysis of piecewise-linear networks, which accounts for the graph-structured architecture rather than simply adopting the complexity bound for fully connected networks (FCNs). Therefore, our estimation analysis does not simply transfer an FCN approximation error bound to SGCNs. We have made these existing theoretical connections and the approximation conditions underlying the convergence rate of Theorem 2 more prominent in the revised manuscript.

AuthorsOpenReview ↗

Reply to Reviewer [3/5]

2c. Correction and clarification of the asymptotic expansion in Eq. (21)

We thank the reviewer for drawing attention to the normalization in Eq. (21). We have revised it accordingly. Specifically, the cross-fitted estimator in Eq. (14) is constructed by averaging the orthogonal scores over all TT observations. For KK equally sized folds, it can be equivalently expressed as

1T∑k=1K∑t∈Ikψ^t,k,\frac{1}{T}\sum_{k=1}^{K} \sum_{t\in I_k}\hat{\psi}_{t,k},

where ψ^t,k\hat{\psi}_{t,k} denotes the score evaluated using nuisance estimates trained outside fold IkI_k. Thus KK determines the cross-fitting partition, while the first-order asymptotic normalization is governed by the total sample size TT. Under the standard DML regularity conditions ensuring asymptotic linearity, the corrected Eq. (21) is

TΨ^−12(μ^−μ∗)=1T∑k=1K∑t∈IkΨ−12[ψ(rt,ft,Gθ∗(xt),Λ(xt))−μ∗]+op(1)→dN(0dμ,Idμ)\sqrt{T}\hat{\Psi}^{-\frac{1}{2}}(\hat{\mu}-\mu^*)=\frac{1}{\sqrt{ T}}\sum_{k=1}^{K}\sum_{t\in I_{k}} \Psi^{-\frac{1}{2}}[\psi(r_t,f_t,G_{\theta^*}(x_t),\Lambda(x_t))-\mu^*]+o_{p}(1)\xrightarrow{d}N(0_{d_{\mu}},\mathbb{I}_{d_{\mu}})

This is consistent with the standard asymptotic linear representation of cross-fitted DML estimators established by Chernozhukov et al. (2018, Theorem 3.1, Eq. (3.10), and Appendix Eq. (A.22)). The correction concerns the normalization and indexing of the displayed asymptotic expansion, rather than the construction of the estimator itself. The cross-fitting estimator defined in Eq. (14), its underlying estimation procedure, the proof of Theorem 3, and the reported empirical results remain unchanged. We appreciate the reviewer's observation, which helps us present the connection between our cross-fitting formulation and the established DML asymptotic theory more precisely.

3. How the FF49 architecture preserves firm-level heterogeneity

In response to this comment, we have added a shared, learned direct characteristic path to the loading head in the revised model. This extension combines the industry context extracted through FF49 aggregation with each firm's own characteristics. Appendix A.14, Eq. (112), specifies the revised architecture, and Table A10 evaluates the contribution of the added path.

Under the complete FF49 graph and the normalization used in Eq. (10), the industry block satisfies A^g=Jng/ng\widehat A_g=J_{n_g}/n_g, where JngJ_{n_g} is the all-ones matrix. The graph branch therefore produces a common representation mgm_g for firms in industry gg. In the revised architecture, the direct characteristic term enters the loading head after graph aggregation:

\operatorname{Readout}_\beta\\!\left( \operatorname{ReLU}\\!\left(m_{g(i)}+Z_iW_s\right)\right).$$ where $m_{g(i)}$ supplies the shared industry context, while $Z_iW_s$ introduces firm-specific information without averaging it across the industry. For firms $i$ and $j$ in the same industry, the difference between their inputs to the activation is $(Z_i-Z_j)W_s$. The added path thus allows their final factor loadings to respond differently to their own characteristics, even though they share the same graph representation. **Individual heterogeneity is represented by a shared nonlinear structural function evaluated at each firm's own characteristics and industry context.** The matrix $W_s$ and the readout parameters are shared across firms. Company-level variation arises from the characteristic inputs. This implements heterogeneous factor exposures without assigning a separate parameter vector to every firm. **Table A10 evaluates the extended architecture over the twelve monthly cross-sections of 2021.** The graph representation is constant within each industry, while the direct path, combined representation, and final loadings have median within-industry variance shares of $0.831$, $0.776$, and $0.745$, respectively. Disabling the direct path reduces the within-industry variance share of the final loadings to approximately $8\times10^{-16}$. Over the same twelve months, stock-level predictive $R^2$ decreases from $0.465\%$ to $0.299\%$. These measurements show that the added characteristic path transmits firm-level variation to the final loadings and contributes to predictive performance. The matched MLP comparison in Table A9 further evaluates the contribution of industry aggregation within the revised architecture. It uses the same direct-path architecture, parameter count, and training protocol, with $\widehat A=I$ replacing FF49 aggregation. Across $K=1,\ldots,6$, mean stock-level predictive $R^2$ is $0.593\%$ with FF49 aggregation and $0.513\%$ with the matched MLP. This comparison shows the predictive value of industry context alongside the firm-specific information provided by the added characteristic path.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.