- Rating
- 2(−1)
- Confidence
- 2(−1)
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the setting in which data from an old policy predict observed transitions accurately but do not identify the consequences of new actions. In recoverable-state linear Gaussian models, it characterizes a compatible dynamics set, uses conditional action residuals to locate missing directions, and allocates an exploration budget to targeted interventions. The motivation is practical, and the theory and controlled experiments support sampling to close specific information gaps. The task relevance of the selected directions and the incremental benefit in more complex settings, however, require stronger evidence.
Critical strengths
-
A coherent identification-to-exploration chain. Theorems 1, 2, 4, and 6 connect state recovery, compatible dynamics sets, counterfactual prediction bounds, and minimum-intervention directions in a way that is clear and checkable within the declared model class.
-
Careful separation of design objectives. Appendix E distinguishes random Gaussian designs, fixed-Gram designs, and expected-information allocation objectives. Propositions 34-35 also analyze estimation error and representation transfer under interventions, avoiding an overstatement that these distinct notions of optimality coincide.
-
Relatively transparent empirical evidence. Appendices G, I, and J include repeated systems, alternative fitters and controllers, assumption-deviation tests, and cases without an improvement. This helps make both the benefits and the failure boundary of directed sampling visible.
Critical weaknesses
-
The allocation criterion is not yet sufficiently task-aware. Section 6 mainly allocates budget by action-effect estimation error, while Proposition 40 explicitly shows that task weights alter the optimal design. The paper does not establish when a low-coverage direction is a control bottleneck rather than an irrelevant but difficult-to-identify direction.
-
The principal fitting comparison may underuse old data. In Appendix G.1, PairedTwoStage estimates the probed directions from new data and then fixes their coefficients, so FullBalanced cannot fully reuse old-data constraints on those coefficients. Although another fitter shows gains, roughly 53 percent of the primary improvement magnitude should be rechecked with information-weighted fusion of old and new data.
-
The complex-task gains over simpler marginal diagnostics are not yet stable. Directed and GlobalMean differ by only 0.14 percentage points in Table 1, and GlobalGap attains the best small-budget mean return in the pixel experiment. The evidence supports the feasibility of directed sampling, but not a consistent additional decision benefit from the conditional-residual diagnostic.
-
The representation-learning extension lacks an empirical bridge. Section 4 and Theorem 22 connect representation error to identification, but Appendix G mainly reports final selection and control outcomes. An analysis varying spectral gaps or alignment errors would clarify the practical value of this extension.
Questions for authors
-
If there are action directions that are poorly covered but irrelevant to the current task, does the method still prioritize them, and can a task-weighted design avoid ineffective interventions?
-
If all methods use a shared estimator that preserves passive information matrices and combines all old and new data, how do the main control gains and the advantage of NoNewData under full coverage change?
-
In real tasks where marginal actions are rich but conditional actions are inadequate, can the conditional-residual diagnostic consistently outperform GlobalMean, GlobalGap, and uncertainty sampling?
2 comments
Thank you for the response and clarifications. However, my concerns have not been fully resolved. I therefore maintain my current rating.
Task relevance, passive-information reuse, and the representation bridge
W1/Q1. Can weak but irrelevant directions receive probes, and how can this be avoided?
Yes. Unweighted identification can spend probes on directions irrelevant to the current task. For a fixed goal, Proposition 40 uses tr(H_task J^(-1)) to prioritize uncertain effects that matter to that goal; unweighted identification supports reuse across unknown future goals.
We added the diagonal rule j_j=(√(h_j/(ηw_j))−λ_j)_+, with η set by the budget. In a common diagonal basis, λ_j is passive information, h_j>0 task importance and w_j>0 action cost. A direction receives zero new information exactly when λ_j≥√(h_j/(ηw_j)); otherwise its task weight determines the amount added. This combines Proposition 40's task weighting with Eq. 18's reuse of passive information. TaskFill estimates task relevance using frozen-model Jacobians; the revision distinguishes this proxy-weight implementation from the fixed-weight design result.
W2/Q2. What changes when all methods reuse old data through a shared estimator?
With every old and new row retained at equal weight, the missing/weak-coverage control comparison changes to favor FullBalanced, and NoNewData's isotropic full-coverage advantage disappears. Following your suggestion, we compared all methods using shared estimators at the original budget of 24. The original approximately 53% reduction belongs to PairedTwoStage. Mean paired control-cost reductions of ConditionalGap versus FullBalanced are:
Positive values favor ConditionalGap. EqualMassJoint assigns each dataset equal total weight; OriginalJoint weights every row equally. Under isotropic full coverage, NoNewData's advantage over FullBalanced changes from +55.29% to −0.87%, slightly favoring FullBalanced with joint fitting.
This comparison also separates estimation from control. Under OriginalJoint, ConditionalGap is closer to the aligned physical action-matrix reference in all 32 cases of each missing/weak regime, with mean relative error reductions of 19.16%/17.45%. Action-effect estimation improves, but the original control reduction does not carry over. The revision specifies both acquisition and final estimation; task-weighted design further accounts for which estimation errors affect the current goal.
W3/Q3. Does conditional screening consistently improve realistic-task decisions over marginal and uncertainty baselines?
The control ranking varies across tasks and budgets; the added natural-data study quantifies gains in action-response prediction. In the added natural-data comparison with joint fitting, Eq. 18 has lower mean action-response error in 8/8 Square/Transport groups versus FullBalanced and GlobalGap, 7/8 versus GlobalEq18, and 3/8 versus uncertainty sampling. Reductions are 3.98–15.70% versus FullBalanced and −.34% to +2.26% versus GlobalEq18 (Table 21).
In PickCube, GlobalMean already identifies the useful near-vertical direction, so conditioning adds little directional information. Directed/GlobalMean success is 69.66%/69.52%; their distance difference is .038 mm, with a 98.75% interval [−.067,.169]. EnsembleDisagreement gives 60.91%. The original feedback-masking result explains when conditioning matters: rich marginal actions can still leave the conditional coverage span unidentified. When a marginal method finds the same useful direction, similar performance is expected.
In pixels, Directed lowers response error against FullBalanced by 12.04%/10.89% at 10/30 episodes. Ensemble has lower response error and GlobalGap the highest return at 10; Directed has lower response error at 30. The new natural-demonstration control checks did not improve success over BC, so their gains are reported as action-response prediction. These comparisons identify the measured benefits while keeping the control endpoint specific to its task and budget.
W4. How is representation error connected to the observed results?
Representation error affects direction recovery through two margins: Theorem 11 bounds encoder MSE by 2ε/γ_rep; Theorem 22 propagates encoder and sampling errors relative to the residual spectral gap. Smaller margins make the diagnostic more sensitive.
Table 5 crosses exact/learned coordinates with true/estimated spans, giving MSE .191523–.193246. The added measurements across the 128 frozen cases give γ_rep=.03943–.09346 and aligned encoder MSE=.15413–.57078. These ranges describe error scales where systems and encoders vary together; Table 5 provides the crossed comparison. The revision connects both measurements to the bounds.