- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the identifiability and sample complexity of estimating linear functionals of discrete latent variables observed through a known stochastic channel. Specifically, the authors consider a latent distribution, an observation matrix, and the observed distribution. The goal is to estimate a target functional from independent observations. The key contributions are: 1. characterizes identifiable linear functionals. 2. for identifiable targets, construct a plug-in estimator and asymptotic efficiency and a matching minimax lower bound.
Critical strengths
-
A unified theoretical characterization of identifiability and estimation efficiency. The paper provides a systematic treatment of identifiable linear functionals, finite-sample estimation error, and asymptotic efficiency under a common observation model. The uniform second-order risk expansion for the feasible plug-in estimator is a useful technical contribution.
-
Theoretical treatment of uncertainty in the observation channel. The extension to finitely many candidate channels is interesting. The authors identify conditions under which channel uncertainty makes estimation impossible and conditions under which adaptation can achieve the same first-order risk as an oracle with knowledge of the channel.
Critical weaknesses
-
My major concern is that the foundational results of the paper are closely related to established results in linear inverse problems, generalized least squares, and semiparametric estimation. For example, the identifiability characterization the construction of minimum-variance feasible weights, and the first-order efficiency bound are classical results, as acknowledged by the authors. Although the paper provides additional results concerning second-order risk, calibration uncertainty, and adaptation across candidate channels, it is not sufficiently clear to me whether these extensions constitute a substantial conceptual or methodological advance beyond existing statistical theory.
-
The paper is motivated by causal representation learning, but the actual setting is substantially more restrictive than the models typically studied in this area. In particular, the framework assumes finite discrete latent and observed spaces, a known deterministic mapping, and a known observation channel. Although the paper considers uncertainty about the channel, this extension still assumes that the true channel belongs to a known finite candidate set. Assuming that the observation process is known removes one of the central challenges of causal representation learning: recovering latent structure and observation mechanisms from data. Moreover, the proposed framework focuses on estimating linear functionals of a latent distribution rather than recovering latent causal variables, their relationships, or causal structure. Consequently, the connection to causal representation learning seems not very strong.
Questions for authors
-
Could the authors clarify which theoretical contributions go substantially beyond existing results in semiparametric estimation, generalized method of moments, and measurement-error models?
-
To what extent can the main identifiability and sample-complexity results be extended to settings where the observation channel is unknown and must be estimated from the same observational data?
Q1. What goes beyond existing results in semiparametric estimation, GMM, and measurement-error models?
The starting point is classical, and the paper says so: which averages are recoverable, the best weights, the first-order variance VA/n, and the normal limit. Our estimator is a version of minimum chi-square, a special case of optimally weighted GMM. Three things go beyond that.
1. The actual mean squared error (MSE) of the estimator one can compute. Classical theory gives a limit: approximately normal with variance VA/n, at one fixed truth. That does not bound the MSE at a given n. We show that the MSE is VA/n plus a remainder of at most C/n2, with one constant C (not in closed form) for all priors bounded away from zero, also when the channel merges hidden states. Why: the estimated weights satisfy the same exact constraint as the true ones, so the error splits exactly into (true weights) times (noise) plus (weight error) times (noise). The second piece has size 1/n, and its cross term with the first is a third moment of the noise, of size 1/n2. A consequence: weights fixed in advance lose at least at order 1/n for some prior, while estimated weights stay within 1/n2 of the best for every prior.
2. The price of an estimated channel when the channel table is not square. Classical results cover square, invertible tables estimated from a validation sample (Rogan and Gladen). We cover more record types than hidden states, when the whole prior is recoverable. With m calibration draws per column, the MSE is VA/n+Ccal/m up to smaller terms, where Ccal is the sum over hidden states j of pj2 times the variance of the best weights under state j. This says how to split a calibration budget, and it checks out: predicted 0.1006, simulated about 0.10.
3. A channel known only up to a list of candidates. The literature treats an uncertain channel mainly by sensitivity analysis and identification bounds; we know of no results there on estimation error. We give three cases. (a) Two candidates can produce the same data but imply different answers: the error stays above a fixed number at every n. (b) The candidates' data distributions are separated by a gap: the data pick the right one, with error probability falling exponentially in n, and the estimator is as accurate, to first order, as with a known channel. (c) The gap is at most about 1/n and the answers still differ: an error of the size of that difference remains for every estimator. The constants of (b) and (c) do not meet; the paper states the open range.
Stated plainly: items 1 and 2 complete classical theory with standard tools. Item 3 is, as far as we know, new in kind.
Q2. What if the channel is unknown and must be estimated from the same data?
With no extra information the results do not extend, and more data does not help. Example: a survey records yes or no, and each record is flipped with an unknown probability f. If 40% of records say yes, this fits a true rate of 40% with no flipping, 25% with 30% flipping, or 0% with 40% flipping. One observed number cannot fix two unknowns. In general, with a completely unknown channel only the frequencies of what is recorded without noise (the label) are recoverable: for any prior with the right label frequencies, some channel reproduces the data exactly.
Extra information that is enough: